This study combines recent developments in computer science and control theory to develop an auto-tuning mechanism for controllers in unmanned aerial vehicles (UAVs). An optimal nonparametric auto-tuning framework is presented that integrates homogeneity theory with deep reinforcement learning (DRL) for UAV controller tuning. The method involves generation of tuning rules (formulas relating the results of tests and controller parameters) by means of a deep neural network (DNN) trained via DRL using the modified relay feedback test (MRFT)-induced oscillations; the trained network can map shape features to tuning rules and the homogeneity-based relationships map the measured amplitude and period of the test oscillations to PD/proportional–integral–derivative (PID) controller gains. Therefore, the controller tuning parameters are obtained from the amplitude and frequency of the test oscillations through the mappings (tuning formulas) that are generated by means of the DNN. The developed controller auto-tuning approach is valid for an arbitrary UAV from the considered class due to the proven homogeneity property of the test-and-tuning mapping. The auto-tuner successfully performs proportional-derivative controller tuning for the UAV dynamics requiring only a fewseconds of the test and a few milliseconds for the controller parameters computation. The effectiveness of the tuning approach is demonstrated by both simulation and experiments; a video demonstration is available in https://www.youtube.com/watch?v=o7_Ubm2h0F4&t=3s

This paper presents a Popov’s criterion gain design framework for boundary-layer sliding-mode control (SMC) employing a hyperbolic tangent nonlinearity. The considered plant is a quadrotor attitude model that captures the dominant rigid-body integrator and mechanical (aerodynamic) lag, while explicitly accounting for parasitic actuator dynamics and input delay. By exploiting Popov’s criterion, explicit sufficient conditions are derived to bound the admissible product of the control gain and boundary-layer slope, ensuring stability of the sliding dynamics in the presence of parasitic effects. The analysis also includes the effect of the equivalent control when added to the control signal. Numerical frequency-domain evaluations and time-domain simulations of representative roll and pitch channels for a quadrotor platform validate the proposed design methodology, demonstrating that the Popov-based bounds provide a practical guideline for selecting stabilizing gains that mitigate chattering while preserving robustness.

This paper presents an equivalent-linearization-based procedure for computing the locus of a perturbed relay system (LPRS) for Lur'e systems. The method uses the describing function to approximate the nonlinearity of the Lur'e system by an amplitude-dependent gain. The amplitude-parameterized equivalent system is linear in nature, thereby enabling the direct application of LPRS to analyze oscillations. The framework is applied to a super-twisting sliding mode control system with actuator dynamics. The predicted oscillation amplitudes and frequencies show close agreement with time-domain simulations.

This work analyzes oscillations in a Chua-like circuit from a new perspective by reformulating the original system as a relay feedback loop that carries a lumped input delay. Leveraging the Locus of Perturbed Relay Systems (LPRS) framework, we identify, for the first time, all possible symmetric periodic orbits—including unstable periodic orbits (UPOs)—that exhibit sign-definite behavior in each half-cycle. The LPRS delivers periodic solutions for all periodic orbits—whether stable or unstable—together with closed-form expressions for periodic waveforms, initial conditions, and orbital stability. The analysis reveals how these solutions emerge, vanish, or transition in stability as the time delay is varied. Through rigorous orbital stability criteria, we map delay-induced bifurcations that mark transitions between chaotic and periodic regimes. Our findings demonstrate that time delay, typically viewed as a parasitic effect, can be harnessed as a tunable parameter for chaos suppression in discontinuous dynamical systems, offering new avenues for controlled oscillations in practical implementations of Chua-like circuits.

We consider the well-known phenomenon in non-ideal sliding-modes, called chattering, which always appears in (real physical) systems due to some unaccounted dynamics, for instance when having inherent sensor and actuator elements in the closed loop. Moreover, any signal’s transport or sampling provide an additional phase lag which can render the relative degree to be higher than two, thus allowing for a harmonic balance solution to exist, and so chattering. Two methods, the describing function (DF) and the locus of a perturbed relay system (LPRS), are used for examining chattering. Our focus is to compare, building up the analysis from previous works, to which extent the chattering is appearing in conventional (i.e. linear) versus the so-called terminal sliding surface design. For the purpose of the latter, we perform linearization at the origin and show how the corresponding loop transfer function varies in gain and affects the harmonic balance solution. The paper reports a detailed experimental case, showing which chattering level can appear and why, especially in light of a not measurable output’s derivative required for the sliding surface, unaccounted delay, and sensor noise. Here DF is used to qualitatively justify chattering, while LPRS delivers its prediction with sufficient accuracy. For the sake of a better exposition of the transient and steady-state responses, the SMC control with both linear and terminal surfaces are also compared in experiments with a loop-optimized standard PID control.

Chattering is an inherent feature of sliding mode control, and its complete elimination is not realistic in practical systems. This paper presents an adaptive sliding mode control scheme for underwater vehicle heave control, where chattering occurs at relatively low frequencies due to the slow plant dynamics. An adaptive law is designed to regulate the relay amplitude to maintain periodic oscillation (chattering) at the plant output, while minimizing its amplitude. External disturbances introduce asymmetry in the relay duty cycle, which is captured through an estimate of the averaged control. This averaged control estimate is obtained using higher-order low-pass filtering of the relay output, with relatively large filter’s time constant. The estimated averaged control is then used to adapt the relay amplitude, maintaining near-symmetric switching without breaking the chattering motion. The proposed method is validated through numerical simulations on an underwater vehicle model, demonstrating that periodic chattering is preserved under the influence of varying external disturbances, while the oscillation amplitude is minimized for a given disturbance, showing enhanced disturbance rejection with minimal chattering amplitude.

This paper proposes a non-parametric automatic tuning method for satellite attitude control that accounts for modeling uncertainties in the inertia and actuator dynamics. The method utilizes a two-relay feedback test, which induces self-sustained periodic oscillations in the system’s output. The existence of the periodic oscillations is analyzed using the describing function method. The profile of these oscillations is used to optimally tune the control parameters through the proposed optimal tuning rules (tuning formulas), with a guarantee on a specific phase margin. The tuning rules are found through numerical optimization for a class of satellites ranging from nanosatellites to large scale satellites, where the attitude dynamics vary depending on the satellite sample. The tuning rules map the oscillations’ amplitude and frequency to the optimal controller gains. The tuning of the controllers happens online in real-time using these tuning rules. The effectiveness of the proposed tuning procedure is evaluated through nonlinear three-axis satellite simulations and validated experimentally using a magnetic-levitation testbed simulating a frictionless space environment.

This work investigates relay-based methods to reveal the frequency domain characteristics of BlueROV2 underwater vehicle. Two relay-based approaches are examined: the Modified Relay Feedback Test (MRFT) and a two-relay (twisting-type) controller. The distinguishing feature of these relay tests is their ability to capture process frequency-response data at a desired phase angle in the Nyquist plot. Approximate equivalence of these tests is also discussed. Influence of additional dynamics on oscillations for these tests is discussed and it is argued that these relay tests have capability to excite both actuator and plant dynamics simultaneously, hence provide a mean to perform full system identification and precise controller tuning. Experiment and simulations of oscillation tests are also presented.

This paper presents a novel approach to improving the generalization capabilities of reinforcement learning (RL) agents for robotic systems with varying physical parameters. We propose the Fuzzy Ensemble of RL policies (FERL), which enhances performance in environments where system parameters differ from those encountered during training. The FERL method selectively fuses aligned policies, determining their collective decision based on fuzzy memberships tailored to the current parameters of the system. Unlike traditional centralized training approaches that rely on shared experiences for policy updates, FERL allows for independent agent training, facilitating efficient parallelization. The effectiveness of FERL is demonstrated through extensive experiments, including a real-world trajectory tracking application in a quadrotor slung-load system. Our method improves the success rates by up to 15.6% across various simulated systems with variable parameters compared to the existing benchmarks of domain randomization and robust adaptive ensemble adversary RL. In the real-world experiments, our method achieves a 30% reduction in 3D position RMSE compared to individual RL policies. The results underscores FERL robustness and applicability to real robotic systems.

Self-excited multi-frequency oscillations in dynamic systems are a rare phenomenon. Dual-frequency oscillations, which may occur as limit cycles with higher-frequency chattering segments, are a peculiar behavior in certain relay systems. Understanding the cause of these oscillations is notably challenging. Other phenomena, such as limit cycles with sliding and chaotic chattering, are different from the present phenomenon under investigation. Their analysis requires specific approaches; the use of the previously developed approaches would not give an explanation of the cause of self-excited dual-frequency oscillations. The paper investigates this complex behavior using analytical methods such as the locus of perturbed relay systems and the describing function, applied in combination in different parts of the proposed analysis. The underlying mechanism of dual-frequency oscillations is studied through the bias function analysis, with application of the above-mentioned methods. Examples exhibiting oscillations of dual frequencies are presented and analyzed to investigate the origins of such complex oscillatory behaviors.

Many Sliding Mode Control (SMC) algorithms are developed and tested on robotic manipulators based on the premise that finite-time convergence can be achieved in practical systems. However, this idealistic approach only works theoretically, as parasitic dynamics (unmodeled dynamics) in real systems lead to non-vanishing oscillations instead of convergence to an equilibrium point. In this work, we argue this fundamental point for the case of robotic manipulators. Two SMC techniques—conventional and Terminal SMC—are analyzed using Describing Function analysis and the Locus of Perturbed Relay Systems on a SOPDT model and a two-link manipulator. Findings confirm that chattering, caused by parasitic dynamics, challenges the feasibility of SMC’s finite-time convergence in real-world applications.

Load transportation through unmanned aerial vehicles (UAVs), such as quadrotors, has a high potential for quick deliveries to locations that are out of the reach of ground vehicles. The complexity of the pick-and-place procedure in such tasks increases if the target location does not have a clearance at the top, necessitating the use of recent learning-based controllers such as reinforcement learning (RL). This article presents a new concept of dual-scale homogeneity, a property defined by scaled magnitudes and time in transformed coordinates that remain independent of system parameters. It demonstrates that applying transformations to achieve this property ensures consistent performance of a quadrotor with a slung load system (QSLS) despite variations in its parameters. Furthermore, it also presents an effective approach to design a parameter-dependent RL policy that homogenizes the QSLS. Unlike plain RL or gain-scheduled proportional-integral-derivative controllers, which confine parameter variations within a predefined range encountered during training or tuning, the developed approach works under large parameter variations, significantly surpassing the performance of traditional controllers. The conducted experiments on load placement in a confined space, utilizing a quadrotor to manage load swing, proved the proposed synergy between the homogeneity transformations and RL, yielding a success rate of 96% in bringing the load to its designated target with a 3-D RMSE of 0.0253 m.

This letter presents VisTune, a method for automatic controller tuning specifically designed for UAVs using vision-based localization (VBL) for position control. In contrast to existing methods that involve manually flying the UAV to collect data for system identification and tuning, our approach leverages relay-based system identification and tuning, which autonomously generates stable oscillations without the need for a stabilizing controller. The entire process concludes within a few seconds. Prior work in vision-based position control of the UAVs often ignores the delay from the perception pipeline, which is quite significant and results in suboptimal tuning and poor control performance. Our approach accounts for perception delay and addresses practical issues, such as varying delays due to varying computation requirements and inevitable estimation errors, which pose challenges in applying relay-based identification and tuning. Typically, VBL system introduces over 100ms of delay, compared to less than 20ms delay when motion capture system is used. Moreover, we show that the perception delay identified by VisTune can be effectively used to temporally advance the feedforward acceleration signal to achieve better tracking performance. Finally, we demonstrate the robustness of the tuned controllers on a trajectory tracking task, reaching speeds of up to 2.1m/s with an RMS control error of only 0.054m. Under wind disturbance of 5m/s, we report an RMSE of 0.116m. A video of the experiments is available at https://youtu.be/hJoT8bn0K0o.

This study introduces an end-to-end Reinforcement Learning (RL) approach for controlling Unmanned Aerial Vehicles (UAVs) with slung loads, addressing both navigation and obstacle avoidance in real-world environments. Unlike traditional methods that rely on separate flight controllers, path planners, and obstacle avoidance systems, our unified RL strategy seamlessly integrates these components, reducing both computational and design complexities while maintaining synchronous operation and optimal goal-tracking performance without the need for pre-training in various scenarios. Additionally, the study explores a reduced observation space model, referred to as CompactRL-8, which utilizes only eight observations and excludes noisy load swing rate measurements. This approach differs from most full-state observation RL methods, which typically include these rates. CompactRL-8 outperforms the full ten-observation model, demonstrating a 58.79% increase in speed and a ten-fold improvement in obstacle clearance. Our method also surpasses the state-of-the-art adaptive control methods, showing an 8% enhancement in path efficiency and a four-fold increase in load swing stability. Utilizing a detailed system model, we achieve successful Sim2Real transfer without time-consuming re-tuning, confirming the method's practical applicability. This research advances RL-based UAV slung-load system control, fostering the development of more efficient and reliable autonomous aerial systems for applications like urban load transport. A video demonstration of the experiments can be found at https://youtu.be/GtGHhOCmy3M .

Adaptation of control parameters in response to external effects or parametric changes is one of the basic capabilities required for high-performance unmanned aerial vehicle (UAV) applications. Adaptation can be done using forced excitations of the unknown UAV dynamics. In this work, we investigate the application of the modified relay feedback test (MRFT) to identify UAV dynamics in outdoor environments. In particular, we focus on UAVs using GPS for positioning and flying in the presence of external wind. The wind is introduced to the nonlinear UAV model and the resultant dynamics are analyzed. MRFT is also introduced to the position control loops to generate oscillations. The experimental results show that MRFT can be used to generate oscillations in outdoor environments. Some recommendations are given for the subsequent use of oscillations for identification and tuning.

In the field of industrial automation, precise control of tower crane operations is critical to ensure safety and efficiency. This paper introduces a novel adaptive sliding mode control (ASMC) strategy aimed at enhancing the control of the trolley motion along the jib of a tower crane. The primary goal of this control system is to effectively suppress the undesirable oscillations—commonly referred to as "chattering"—that often cause damage to the equipment. Such chattering not only compromises the safety of crane operations but also affects the longevity of the equipment. The proposed ASMC strategy incorporates adaptive mechanisms that adjust the controller parameters in real-time. This adaptability ensures robust performance and significantly mitigates chattering without sacrificing the system’s responsiveness. Furthermore, the developed controller has been successfully deployed and rigorously tested on an experimental setup of a real-world tower crane system. The experimental results demonstrate the efficiency of the ASMC in reducing chattering, thereby validating its potential for real-world applications in tower crane systems. This research contributes to the ongoing efforts in automation technology by providing a reliable solution to a prevalent challenge in crane operations. A video demonstration of the experiments can be found at https://youtu.be/-TRNeVKn7KY.

Chattering in Sliding Mode (SM) control is known as fast self-excited periodic motions (oscillations) that occur due to the combination of two factors: nonlinearities that are not Lipschitz continuous and additional dynamics of actuators and sensors (the dynamics not accounted for in SM control system design). It is found in the present research that under certain combinations of parameters of the discontinuous homogeneous sliding mode controller and actuator, the chattering is manifested not as periodic but as chaotic oscillations. This work investigates the phenomenon of chaotic chattering through the Locus of Perturbed Relay Systems (LPRS) method and the formulated conditions for the periodic solutions found through the LPRS to manifest themselves as periodic motions. Adherence to or violation of these conditions is correlated with the occurrence of periodic or chaotic chattering. The particular controller investigated is a Homogeneous Sliding Mode Controller. Bifurcation points in terms of the time constant of the actuator and the relay amplitude are determined.

Abstract Trained deep reinforcement learning (DRL) based controllers can effectively control dynamic systems where classical controllers can be ineffective and difficult to tune. However, the lack of closed‐loop stability guarantees of systems controlled by trained DRL agents hinders their adoption in practical applications. This research study investigates the closed‐loop stability of dynamic systems controlled by trained DRL agents using Lyapunov analysis based on a linear‐quadratic polynomial approximation of the trained agent. In addition, this work develops an understanding of the system's stability margin to determine operational boundaries and critical thresholds of the system's physical parameters for effective operation. The proposed analysis is verified on a DRL‐controlled system for several simulated and experimental scenarios. The DRL agent is trained using a detailed dynamic model of a non‐linear system and then tested on the corresponding real‐world hardware platform without any fine‐tuning. Experiments are conducted on a wide range of system states and physical parameters and the results have confirmed the validity of the proposed stability analysis (https://youtu.be/QlpeD5sTlPU).