The method [K]control of Adaptive Multiple-timescale Systems (KAMS) has been used as a method of adaptive control for systems with states that evolve at vastly different rates and with uncertain parameters. Prior research on KAMS has shown a relationship between the asymptotic behavior of the reference models and the behavior of the full-order closed-loop system. Poor reference model design can cause steady-state error or instability even if that reference model is acceptable in a single timescale context, due to adverse feedback between the reference models and the fast-state manifold. This paper simplifies KAMS control design and provides insight into multiple timescale reference models by identifying design considerations for the fast and slow reference models. For fast reference models, it is shown that the stability proofs used in KAMS control design necessarily require the manifold to be an asymptotically stable equilibrium of the fast reference model, despite prior work that allowed other possibilities. In the case of a slow reference model that is not asymptotically stable to its reference signal, it is common for the inner connection conditions to be dependent on the reference signal. This can cause the interconnection conditions to be ill-formatted for the application of the stability proofs used in KAMS control design. This paper addresses that case and proposes the use of Raffoul's condition as a method to prove stability in the sense of Lyapunov. A linear quadrotor attitude stabilization problem is used as a framework for the slow reference model analysis. Results presented in the paper show that 1) the asymptotic stability of the fast reference model is a necessary condition for the KAMS stability proofs used in prior work and 2) Raffoul's condition is a sufficient condition for Lyapunov sense stability even when the inner connection conditions are ill-formatted for the KAMS stability proofs used in prior work.

Reinforcement learning has been used to train autonomous single-agent and multi-agent systems to solve complex problems in the aerospace domain. This type of machine learning is a long process, often requiring training to run millions of time steps to achieve practical solutions. Training environments are often simplified to expedite training, such as using reduced vehicle dynamics and environmental abstractions to remove dependency on visual-based graphics. However, these complexities may be favorable when realism and accuracy are prioritized over run-time during full autonomous system validation. This paper introduces and develops an architecture that facilitates the transition from a reduced training environment to more realistic simulations. Modularity is emphasized to reduce implementation time and scalability to support large multi-agent systems. The proposed system leverages Unreal Engine 5 for high-fidelity graphics, optimized performance, and developer-oriented workflow to create a simulation for demonstrating and validating multi-agent systems for real-world deployment. The integration of Software-in-the-Loop (SITL) and Hardware-in-the-Loop (HITL) further validate the simulation framework, allowing for detailed testing and adjustment phases. Results presented in the paper investigate case studies on the validation for deployment of a single-agent autonomous tracking UAS and extend the proposed framework to an auction-based multi-agent coordination scenario.

Inlet unstart on hypersonic aircraft causes a rapid and dangerous loss of thrust. Fortunately, proper control design can help prevent inlet unstart. This paper demonstrates how [K]control of Adaptive Multiple timescale Systems (KAMS) can effectively address this challenging problem. First, a multiple-timescale model of a hypersonic aircraft is developed to facilitate the control law design. Then, a KAMS controller is designed using Adaptive Nonlinear Dynamic Inversion to stabilize the reduced subsystems and Sequential Control is used to fuse the control signals for reduced subsystems. The closed-loop system is proven to be stable despite weak non-minimum phase effects. KAMS provides stability guarantees that are more rigorous than prior work and also provides insights into the system's underlying physics. Numerical results presented in the paper show that KAMS can effectively prevent inlet unstart and mitigate uncertainty using angle-of-attack regulation.

Singularly perturbed systems are a class of mathematical systems that are not well approximated by their limits and can be used to model plants with multiple fast and slow states. Multiple-timescale systems are very common in engineering applications, but adaptive control can be sensitive to timescale effects. Recently a method called [K]control of Adaptive Multiple-timescale Systems (KAMS) has shown improved performance and increased robustness for singularly perturbed systems, but it has only been studied on systems using adaptive control for the slow states. This article extends KAMS to the general case when adaptive control is used to stabilize both the slow and fast states simultaneously. This causes complex interactions between the fast state reference model and the manifold to which the fast states converge. It is proven that under certain conditions the system still converges to the reference model despite these complex interactions. This method is demonstrated on a nonlinear, nonstandard, numerical example.

Machine learning has long been discussed as a candidate for facilitating autonomous multiagent vehicle coordination. Many methods of autonomous multiagent coordination have been proposed, however few if any solutions take into account realistic communication challenges. By using machine learning on multiple levels, and a self organizing hierarchical system, an autonomous, pseudo decentralized, heterogeneous, system can dynamically complete tasks without being fully connected. This method will be investigated and demonstrated on a simple, proof of concept rotorcraft simulation. Initial results show that a parallel auction coordination framework can be used to organize multiple heterogeneous agents with different sensors, movement modalities, and controllers to complete a task requiring multiple agents. The auction framework is independent of individual agents and has been utilized in this paper by a combination of a reinforcement learning trained agent and two potential optimally controlled agents to complete a task.

Current autonomous unmanned aerial systems (UASs) commonly use vision-based landing solutions that depend upon fiducial markers to localize a static or mobile landing target relative to the UAS. This paper develops and demonstrates an alternative method to fiducial markers with a combination of neural-network-based object detection and camera intrinsic properties to localize an unmanned ground vehicle (UGV) and enable autonomous landing. Implementing this visual approach is challenging given the limited compute power on board the UAS, but it is relevant for autonomous landings on targets for which affixing a fiducial marker a priori is not possible or not practical. The position estimate of the UGV is used to formulate a landing trajectory that is then input to the flight controller. Algorithms are tailored toward low size, weight, and power constraints, as all compute and sensing components weigh less than 100 g. Landings were successfully demonstrated in both simulation and experimentally on a UGV traveling in both a straight line and while turning. Simulation landings were successful at UGV speeds of up to 3.0 m/s, and experimental landings at speeds up to 1.0 m/s.

Previous approaches for small fixed-wing unmanned air systems that carry strapdown rather than gimbaled cameras achieved satisfactory ground target tracking performance using both standard and deep reinforcement learning algorithms. However, these approaches have significant restrictions and abstractions to the dynamics of the vehicle, such as constant airspeed and constant altitude, because the number of states and actions was necessarily limited. Thus, extensive tuning was required to obtain good tracking performance. The expansion from 4 state–action degrees of freedom to 15 enabled the agent to exploit previous reward functions that produced novel yet undesirable emergent behavior. This paper investigates the causes of and various potential solutions to undesirable emergent behavior in the ground target tracking problem. A combination of changes to the environment, reward structure, action space simplification, command rate, and controller implementation provides insight into obtaining stable tracking results. Consideration is given to reward structure selection and refinement to mitigate undesirable emergent behavior. Results presented in the paper for a simulated environment of a single unmanned air system tracking a randomly moving single ground target show that a soft actor–critic algorithm can produce feasible tracking trajectories without limiting the state space and action space, provided that the environment is properly posed.

Tracking the motion of an evasive, or even hostile, maneuvering ground target based on no percepts other than aerial images is a challenge that is relevant to military and commercial applications. This paper investigates the problem of tracking such a target using a multirotor unmanned air system with a non-gimballed optical sensor. The fixed sensor makes the problem challenging because the vehicle must use its dynamics and therefore attitude to change the sensor's field of view. A Soft Actor-Critic reinforcement learning controller is developed to address the maneuvering ground target tracking problem for various target movement types including a sinusoidal motion, random motion, and evasive motion. The addition of ground occlusions which block the agent's view of the target is also investigated. This is challenging because the reinforcement learning agent does not have information about the location or the existence of occlusions, but must still generate actions which track the target. Results in a simulated environment demonstrate the effectiveness of the learned policy. Consistent indefinite tracking is achieved with the mean distance of the target from the center of the image varying from 23 to 27 pixels for all motion types tested. Selection of the command rate, the rate at which the agent can generate actions, is also investigated. Faster command rates from the learned policy are shown to be less effective for tracking a maneuvering ground target since the agent needs to perform a greater number of correct sequential actions. Finally, the agent is able to successfully track a randomly moving target with the presence of occlusions.

This paper presents a summary of system identification flight testing and results for a variety of large and small fixed-wing and multirotor unmanned air systems at Texas A&M University from 1999 to 2023. The six different types of vehicles range from a large powered parafoil, to a fixed-wing vehicle with synthetic jet-actuated roll control effectors, to a radially asymmetric multirotor, to large and small fixed-wing vehicles, and to a Steppe eagle. The observer/Kalman filter identification algorithm is used to generate linear time-invariant state-space models, and the results for both near-real-time online model generation and postflight offline model generation are presented. The use and efficacy of a variety of test input types and their sensitivity to exogenous inputs such as turbulence, in addition to identified model evaluation and selection criteria, are discussed. Several generations of low size, weight, power, and cost flight-test instrumentation including the Developmental Flight-Test Instrumentation data acquisition package are also presented. Challenges that arose from the flight-testing campaigns along with solutions are highlighted in the paper.

The cyber threat to aerospace systems has been growing rapidly in recent years with several real-world and experimental cyberattacks observed. This growing threat has prompted investigation of cyberattack and defense strategies for manned and unmanned air systems, spacecraft, and other aerospace systems. The work in this paper seeks to further understand these attacks by introducing and developing a novel cyberattack for autonomous aerospace systems. The problem faced by the attacker is posed and discussed analytically using false data injection of state measurements to exploit the vehicle’s onboard controller to take control of the system. It is shown that the attacker can utilize traditional control techniques to exert control over the system and eliminate the control of the victim by intercepting and modifying the vehicle’s measurement data. The attacker is able to accomplish this objective without any prior knowledge of the system’s plant, controller, or reference signal. The attack is demonstrated on the elevator-to-pitch-attitude-angle dynamics of a Cessna T-37 aircraft model. It is shown to be successful in eliminating the victim’s control influence over the system and driving the system to its own target state.

Multiple-timescale systems are a noteworthy class of dynamic systems that can be modeled with singularly perturbed differential equations. Adaptive control has not been studied in the context of singularly perturbed plants. This paper introduces and evaluates three methods of adaptive control for multiple-timescale systems. Each method is a framework that is valid for a wide class of adaptive control methods. Full-order adaptive control (FOAC) applies adaptive control to the system as a whole. It is straightforward but can be sensitive to timescale effects. Reduced-order adaptive control (ROAC) applies adaptive control to either the fast or slow modes only. This simplifies synthesis but can also constrain the range of valid timescale separation. [K]Control of Adaptive Multiple-Timescale Systems (KAMS) fuses two adaptive control signals using multiple-timescale techniques. KAMS takes advantage of model reduction unlike FOAC, and allows for unstable fast dynamics unlike ROAC. Generalized formal definitions, stability criteria, and examples are developed and presented for each method. Results presented in the paper for the control of a Boeing 747-100/200 on approach show that KAMS has a desirable blend of performance and robustness because each reduced-order model is stabilized separately.

Adaptive control for non-minimum phase systems is a challenging problem. This paper proposes a method of adaptive control for systems that may be both nonlinear and non-minimum phase. This is accomplished by exploiting time scale separation between the internal and external dynamics. The original non-minimum phase control problem is reduced to two minimum phase control problems through a time scale analysis. The resulting adaptive control signals are fused via multiple time scale control techniques. Singular perturbation theory is used to prove the stability and convergence of the full-order system as an extension of the stability and convergence of the two reduced-order systems. The effectiveness of this method is validated on a nonlinear example system.

Multiple time scale systems are a noteworthy class of dynamical systems. Adaptive control has not been studied in the context of multiple time scale systems because it is not robust to time scale separation. This paper introduces and evaluates three methods of adaptive control for multiple time scale systems. Each method is a framework that is valid for a wide class of adaptive control methods. Full-Order Adaptive Control (FOAC) applies adaptive control to the system as a whole. It is straightforward but sensitive to time scale uncertainty. Reduced-Order Adaptive Control (ROAC) applies adaptive control to either the fast or slow modes only. This simplifies synthesis but can also constrain the range of valid time scale separation. [K]Control of Adaptive Multiple Time Scale Systems (KAMS) fuses two adaptive control signals using multiple time scale techniques. Generalized formal definitions, stability criteria, and examples are developed and presented for each method. Results show that [K]Control of Adaptive Multiple Time Scale Systems has the best performance because each reduced-order model is stabilized separately and because the fast dynamics converge to the manifold more quickly than the other methods. It also takes advantage of model reduction and relaxes the stability requirements of the other methods.