Small satellite technology performance and capability have advanced at a pace that seems to increase exponentially even as the size, weight, and power (SWaP) needed to yield such performance have decreased. Taken in context with the advent of a proliferated low Earth orbit fueled by plummeting launch costs, promises of low-latency, continuously connected, pervasive communications and remote sensing spacecraft are no longer science fiction, they can be engineered fact.
Technological advancement also means an increase in complexity of individual spacecraft, where a single unit no longer hosts a single payload, but may hold several instruments and associated supporting subsystems that all compete for the same resources. It no longer waits for perfect timing before gathering or downlinking data, it needs to provide constant data at high downlink speeds multiple times each orbit or even continuously. An individual spacecraft may often be required to act as part of a swarm of spacecraft thereby enhancing the data product by increasing both spatial and temporal resolution. These increases in performance requirements are enabled by increased system-of-systems complexity, which means there are more controllable signals and systems that must be managed.
Furthermore, as the number of connected systems increases, the complexity of managing these swarms or constellations taxes ground operators and support infrastructure. Commanding large networks of spacecraft has already leveraged autonomy, both on-board and on the ground, but moving forward such autonomy must advance to the point of “minimal human oversight” before enormous constellations of satellites can be maintained in a cost-effective manner.
Generally, the decision space dimensionality, or the number of logic rules needed to completely cover input-output combination mappings, increases exponentially with an increase in the input space and linearly with outputs. It is impractical to design an autonomous system to cover all possible combinations without an enormous amount of data and precise knowledge of the operating environment, possible failure modes, and other perturbances that will inevitably degrade the system performance. The list of unknown unknowns is too long and unpredictable to investigate in full. If it were easy to predict, designers would simply account for the scenario.
A Hierarchical Ensembles of Autonomous Decision Systems (HEADS) system with a recursive ensemble weighting update is proposed. The system is built on fuzzy logic leading to an understandable and tractable logic design that leverages subject matter experts to design system operations. The hierarchical structure enables multi-layered logic for granular control and decisions incorporating inferred information. The control output from each ensemble is a mixture from independently trained fuzzy systems processed through a gating network. The gating network weights are updated recursively. Each expert uses a subset of the input space to minimize per-expert complexity and support ensemble robustness under uncertain or evolving state realizations and operating environments. Finally, autonomy based on fuzzy systems offers the potential for increased human comprehension of an agent's status and decision logic.
Additional advantages will be set forth in part in the description that follows, and in part will be obvious from the description, or may be learned by practice of the aspects described below. The advantages described below will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims. example
It is to be understood that both the foregoing general description and the following detailed description are examples and explanatory only and are not restrictive of the systems and methods as claimed.
The accompanying figures, which are incorporated in and constitute a part of this specification, illustrate several aspects of the disclosure, and together with the description, serve to explain the principles of the disclosure.
As shown by the agent 110A, each agent 110 may include a control system 115, one or more sensors 120, and an autonomous decision system 130. The sensors 120 may include a variety of sensors 120 such as cameras, thermometers, radar sensors, lidar sensors, etc. Any type of sensor 120 may be supported. The autonomous decisions system 130 may be implemented using one or more general purpose computing devices such as the computing device 900 illustrated with respect to
The control system 115 may control the functioning of the associated agent 110 including functions related to navigation (e.g., direction, velocity, braking), agent state (e.g., sleep, reboot, or reset), as well as the operation of the various sensors 120 (e.g., activate, deactivate, and power mode).
The autonomous decisions system 130 may receive one or more input signals 140, and in response to the input signals 140, may generate control signals 150. The control signals 150 generated by the autonomous system 130 may then be used by the control system 115. For example, a control signal 150 may indicate that a camera sensor 120 should be disabled. In response to receiving the control signal 150, the control system 115 may disable the camera sensor 120.
The input signals 140 may include a variety of signals 140 including signals generated by the sensors 120 (e.g., current location, velocity, acceleration, temperature), signals 140 generated by the control system 115 (e.g., status of the various sensors 120, current location, power level). The input signals 140 may also include signals 140 received from other autonomous agents 110. Any type of signal may be supported.
Ensembles 205 are a group of experts 210 separately trained or otherwise managed that are tasked with addressing the same question. The outputs of all experts 210 are combined to arrive at a single output 215 which has the benefit of a variety of supporting experts 210 that taken together may provide an answer closer to the truth than any single expert 210 could do. With enough individual experts 210 trained across the problem domain, some are bound to show excellent performance and some will be poor performers. The goal is to train the ensemble 215 such that the overall performance is acceptable across the range of anticipated operating environments or states.
As shown in the ensemble 205A, each ensemble 205 may include one or more experts 210 (e.g., the experts 210A, 210B, and 210C) that each generate a value based on some or all of the input signals 140 using fuzzy logic. The ensemble 205 may then use the values generated by some or all of the experts 210 to generate the corresponding output 215. The operation of the experts 210 is described further below. Note that input signals 140 as described herein may further include the output 215 of other ensembles 205.
Note the possible arrangements of ensembles and experts is not limited to those shown and described in
The structure of the proposed autonomous decision system 130 combines the benefits of ensemble systems using fuzzy logic with the multi-objective decision making capability of a hierarchical system. Applied to the autonomous operation of an agent 110 with multiple payloads and objectives such as a spacecraft, this system 130 provides accurate and robust performance when faced with degraded subsystems or competing objectives.
The proposed structure of hierarchical ensembles of fuzzy experts can be extended across multiple systems to account for cooperative operations among teams or swarms of satellites. Assuming a minimal amount of crosslinked data transfer, such coordination is straightforward and enables a more robust network that can self-heal from subsystem or spacecraft failures. This can be done without centralized control, which reduces network susceptibility to single-node failure. Finally, having this capability on-board each spacecraft reduces the burden on ground infrastructure and operators, meaning increased efficiency and cost-effectiveness.
In some embodiments, the experts of 210 of the autonomous decision system 130 may use fuzzy logic. For purposes of background, Fuzzy Logic is now described.
Fuzzy Logic can be seen as an extension of Boolean logic that better captures uncertainty. The Fuzzy inference System (FIS) has been extended to be adaptive and combined with neural networks to increase performance. Fuzzy Logic is a non-probabilistic approach to addressing uncertainty where we are concerned more with the degree to which something happens opposed to whether something happens.
A fuzzy set maps part of an input variable domain to [0, 1] and is represented by a linguistic variable. An activation function (which can be continuous or discrete) and its parameters are chosen to enable sufficient coverage and overlap of the input variable domain into the fuzzy domain.
Output fuzzy sets are similar to input fuzzy sets in definition and implementation. The main difference is that input fuzzy sets are used as rule antecedents and output fuzzy sets as rule consequents.
The number of fuzzy sets used to represent an input or output can vary between fuzzy expert systems. For example, one expert might divide an input into three fuzzy sets, another expert might use seven fuzzy sets. This is a design choice driven by the granularity requirements of the rule set in use by the specific fuzzy expert. This variability is one way the autonomous decision system 130 increases robustness, since each expert is likely to not perform well over different sections of the input space.
In some embodiments, a rule may be written in the form of standard logic using AND (∧), OR (∨), NOT (¬) and THEN (→) terms and parenthetical grouping to evaluate into a rule consequent activation. For input fuzzy sets A, B, C, and D, and output fuzzy sets Y and Z, an example a rule set might read:
Fuzzy logic rule evaluation combines fuzzy set activation levels for all antecedent terms. This process is referred to as “fuzzy reasoning” as it generalizes the application of logic inference. The value resulting from fuzzy reasoning is the output fuzzy set, or rule consequent, activation.
Rule evaluation may apply to one or more antecedent terms. When multiple antecedents are considered, fuzzy logic operators dictate value propagation:
Parenthetical statements group terms:
The resulting activation level is then used to bound the output fuzzy set, or consequent, activation. This process is where fuzzy expert systems handle the uncertainty of “to what degree,” as an output inference can be considered “true” to the extent that the consequent activates.
After rule evaluation, the fuzzy expert system consists of one or more output fuzzy set activations. Any particular output fuzzy set can have zero or more component activations that need to be combined via T-norm or T-conorm before the fuzzy expert system can provide a usable crisp output.
The T-norm operator is a function T: [0, 1]×[0, 1]→[0, 1] that defines fuzzy intersection. Common T-norms include the minimum and algebraic product. The T-conorm (or S-norm) similarly maps S: [0, 1]×[0, 1]→[0, 1]. Common S-norms include the maximum and algebraic sum S(a, b)=a+b−ab. The T-norm and T-conorm provide the basis for combining fuzzy sets via intersection and union, respectively.
The final duty of a fuzzy expert system is to produce a crisp output value built from the aggregated output fuzzy sets. This process is called “defuzzification” as it takes fuzzy values as input and gives a crisp output. Common methods include: Centroid of Area, Bisector of Area, Mean of Maximum, Smallest of Maximum, and Largest of Maximum. As many of these operations consist of calculating areas or means of activation-limited output fuzzy sets, the computational burden may be significant. The selection of a defuzzification operation may depend on the output variable and whether the system must be lenient or conservative when reasoning.
Several fuzzy inference systems have been defined that provide the T-norm and T-conorm selections for both rule antecedent combination and rule consequent aggregation. Some of these benefit from crisp outputs after consequent aggregation, reducing the computational burden from defuzzification.
The expert 210 further includes a plurality of rules 330. As shown, each rule 330 receives one or more sub-signals, applies one or more logical operators to the received sub-signals using fuzzy logic as described above. Each rule 330 may be created by a user or administrator. As shown, each rule 330 is written using one or more logical operators. Using logical operators allows one or more human reviewers to quickly and easily create rules 330 as well as understand the function of each existing rule 330. However, other types of syntax may be used for the rules 330.
In addition, in some embodiments, the sub-signals and terms and operators of each rule 330 (referred to as parameters) may be adjusted or optimized to improve the performance of the expect 210. In one embodiment, the parameters may be adjusted using a genetic algorithm. Other methods for optimizing parameters may be used.
The output of each of the rules 330 may be received by a defuzz module 340. The defuzz module 340 may perform one or more defuzzification operations as described above. The performed defuzzification operations may be selected by a user or administrator. The defuzz module 340 may generate the output 315 of the expert 210, which may include a confidence value.
Continuing to
The ensemble 205 may further include a gating network 410 that combines the output signals 315 from each of the experts 210 using a function 420. The result of the function 420 is the output 215. An example function 420 is the following softmax function where Ij is the output 310 of a particular expert 210 of a total of K experts 210 and exp is the exponential function:
The softmax function gives larger weight to relatively larger values in a set. However, the ensemble 205 does not need to produce the output 215 based on the largest output 315 of a single expert 210. Larger outputs 315 do not necessarily mean more correct.
Instead the ensemble 205 may use the confidence value associated with each output 315 to weigh each output 315 in the softmax equation. The output 215 is then the weighted sum of the outputs 315 of the experts 210. As may be appreciated, a single expert 210 with a higher confidence in its output 315 will impact the output 215 of the ensemble 205 to a greater extent than the other experts 210. Similarly, if all of the experts 210 have similar confidence value then they will each influence the output 315 is a similar manner.
In addition, some or all of the output 215 is fed back to all of the ensembles 205 as part of the input signals 140. As shown, the autonomous decision system 130 first adds some amount of delay 505 to the output 215. Accordingly, the output 215 fed back into the ensembles 205 may represent the output 215 generated by the ensembles 205 in response to the previous set of input signals 140. Accordingly, the rules 330 used by the experts 210 in each ensemble 205 can consider a variety of input including the current output 215 of other ensembles 205 as well as the previous output 215 (i.e., with the delay 505) of other ensembles 205.
At 605, a plurality of input signals is received. The plurality of inputs may be the input signals 140 and may be received by the autonomous decision system 130. In some embodiments, the input signals 140 may include signals generated by one or more sensors 120 associated with the autonomous agent, one or more signals generated by an ensemble of the autonomous decision system 130, and one or more signals received from other autonomous agents 110 or other external sources (e.g., control signals 150).
At 610, control signals are generated. The control signals 150 are generated by the autonomous decision system 130. In some embodiments, the autonomous decision system 130 may include an ensemble 205 for each of the plurality of control signals 150. Each ensemble 205 may generate a respective one of the control signals 150.
At 615, operation of the autonomous agent is controlled based on the control signals. The control system 115 may control the autonomous agent 110 based on the control signals 150. For example, the control system 115 may activate or deactivate one or more sensors 120, or may perform one or more navigation maneuvers (e.g., accelerate, decelerate, turn).
At 705, a plurality of input signals is received. The plurality of inputs may be the input signals 140 and may be received by the ensemble 205. The plurality of signals may be a subset of the input signals 140 or all of the input signals received by the corresponding autonomous decision system 130. The ensemble 205 may be one of a plurality of ensembles 205 associated with the autonomous decision system 130, with each ensemble 205 generating a different control signal 150 of the plurality of control signals 150. The ensemble 205 may include a plurality of experts 210.
At 710, a signal is generated by each expert of the ensemble. Each expert may generate a signal or output 315. The output 315 generated by each expert 210 may be associated with a confidence that reflects how confident the expert is that the output 315 is correct.
At 715, the generated outputs from each expert are combined to generate a control signal. The generated outputs 315 from each expert 210 may be combined by the ensemble 205. In some embodiments, each output 315 may be assigned a weight that is based on its associated confidence. The weighted outputs 315 may be combined using a softmax function, for example.
At 805, a plurality of input signals is received. The plurality of inputs may be the input signals 140 and may be received by the expert 210. The plurality of signals may be a subset of the input signals 140 or all of the input signals received by the corresponding autonomous decision system 130. The expert 210 may include a plurality of fuzzy logic rules 330.
At 810, each rule is evaluated to generate a fuzzy logic value. The rules 330 may be evaluated by the expert 210.
At 815, a defuzzification process is applied to the generated fuzzy logic values to generate an output signal. The defuzzification process may be applied by the expert and may convert all of the fuzzy logic values generated by the rules 330 into an output 315.
Numerous other general purpose or special purpose computing devices environments or configurations may be used. Examples of well-known computing devices, environments, and/or configurations that may be suitable for use include, but are not limited to, personal computers, server computers, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, network personal computers (PCs), minicomputers, mainframe computers, embedded systems, distributed computing environments that include any of the above systems or devices, and the like.
Computer-executable instructions, such as program modules, being executed by a computer may be used. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Distributed computing environments may be used where tasks are performed by remote processing devices that are linked through a communications network or other data transmission medium. In a distributed computing environment, program modules and other data may be located in both local and remote computer storage media including memory storage devices.
With reference to
Computing device 900 may have additional features/functionality. For example, computing device 900 may include additional storage (removable and/or non-removable) including, but not limited to, magnetic or optical disks or tape. Such additional storage is illustrated in
Computing device 900 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by the device 900 and includes both volatile and non-volatile media, removable and non-removable media.
Computer storage media include volatile and non-volatile, and removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Memory 904, removable storage 908, and non-removable storage 910 are all examples of computer storage media. Computer storage media include, but are not limited to, RAM, ROM, electrically erasable program read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information, and which can be accessed by computing device 900. Any such computer storage media may be part of computing device 900.
Computing device 900 may contain communication connection(s) 912 that allow the device to communicate with other devices. Computing device 900 may also have input device(s) 914 such as a keyboard, mouse, pen, voice input device, touch input device, etc. Output device(s) 916 such as a display, speakers, printer, etc. may also be included. All these devices are well known in the art and need not be discussed at length here.
It should be understood that the various techniques described herein may be implemented in connection with hardware components or software components or, where appropriate, with a combination of both. Illustrative types of hardware components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc. The methods and apparatus of the presently disclosed subject matter, or certain aspects or portions thereof, may take the form of program code (i.e., instructions) embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium where, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the presently disclosed subject matter.
Although example implementations may refer to utilizing aspects of the presently disclosed subject matter in the context of one or more stand-alone computer systems, the subject matter is not so limited, but rather may be implemented in connection with any computing environment, such as a network or distributed computing environment. Still further, aspects of the presently disclosed subject matter may be implemented in or across a plurality of processing chips or devices, and storage may similarly be effected across a plurality of devices. Such devices might include personal computers, network servers, and handheld devices, for example.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
This application claims priority to U.S. Provisional Patent Application No. 63/364,042, filed on May 3, 2022, and titled “HIERARCHICAL ENSEMBLE OF AUTONOMOUS DECISION SYSTEMS FOR AUTOMATED REASONING.” The disclosure of which is hereby incorporated in its entirety.
Number | Date | Country | |
---|---|---|---|
63364042 | May 2022 | US |