This paper provides a rigorous derivation for what is known in the literature as the Lie bracket approximation of control-affine systems in a more general and sequential framework for higher-orders. In fact, by using chronological calculus, we show that said Lie bracket approximations can be derived, and considered, as higher-order averaging terms. Hence, the theory provided in this paper unifies both averaging and approximation theories of control-affine systems. In particular, the Lie bracket approximation of order (n) turns out to be a higher-order averaging of order (n+1). The derivation and formulation provided in this paper can be directly reduced to the first and second-order Lie bracket approximations available in the literature. However, we do not need to make many of the unproven assumptions provided in the literature and show that they are in fact natural corollaries from our work. Moreover, we use our results to show that important and useful information about control-affine extremum seeking systems can be obtained and used for significant performance improvement, including a faster convergence rate influenced by higher-order derivatives. We provide multiple numerical simulations to demonstrate both the conceptual elements of this work as well as the significance of our results on extremum seeking with comparison against the literature.

Extremum Seeking Control (ESC) systems possess the appealing capability of autonomously driving a dynamical system toward the extremum point of an objective function, even when the mathematical expression of the function is unknown a priori and only its measurements are accessible. Such systems, especially with control affine structures, have recently gained significant attention with application to various domains, including but not limited to multi-agent systems of unmanned vehicle/aerial systems. The problem however in applying ESC -- despite its appeal -- in unmanned systems lies in the fact that ESC systems are highly oscillatory and can hardly be designed to have no steady-state oscillations. This in return results in ESC systems being impractical to apply on unmanned systems. Very recent developments have made it possible to attenuate (by design) the oscillations of control affine ESC systems and achieve a steady state free of oscillations. This was done by using a novel concept of Geometric-based Kalman Filtering (GEKF) which allowed accurate estimation of the unknown gradient in control affine ESC systems. In this paper, we explain the novel GEKF and how it can be applied and implemented. Then, we apply for the first time a control affine ESC design that is applicable to multi-agent vehicle or drone (UAV) problems; this ESC design allows the agents to converge to the extremum point of the objective function with attenuated oscillations and free of steady-state oscillations.

For centuries, soaring birds -- such as albatrosses and eagles -- have been mysterious and intriguing for biologists, physicists, aeronautical/control engineers, and applied mathematicians. These fascinating biological organisms have the ability to fly for long-duration while spending little to no energy. This flight technique/maneuver is called dynamic soaring (DS). For biologists and physicists, the DS phenomenon is nothing but a wonder of the very elegant ability of the bird's interaction with nature and using its physical ether in an optimal way for better survival and energy efficiency. For the engineering community, it is a source of inspiration and an unequivocal promising chance for bio-mimicking. In literature, significant work has been done on modeling and constructing control systems that allow the DS maneuver to be mimicked. However, mathematical characterization of the DS phenomenon in literature has been limited to optimal control configurations that utilized developments in numerical optimization algorithms along with control methods to identify the optimal DS trajectory taken (or to be taken) by the bird/mimicking system. In this paper, we provide a novel two-layered mathematical approach to characterize, model, mimic, and control DS in a simple and real-time implementation. The first layer will be a differential geometric control formulation and analysis of the DS problem. The second layer will be a linkage between the DS philosophy and a class of dynamical control systems known as extremum seeking systems. We believe our framework captures more of the biological behavior of soaring birds and opens the door for geometric control theory and extremum seeking systems to be utilized in systems biology and natural phenomena. Simulation results are provided along with comparisons with powerful optimal control solvers to illustrate the advantages of the introduced method.

Dynamic soaring is a remarkable flight strategy employed by soaring birds like albatrosses to harness energy from the atmospheric wind gradient. This strategy is so efficient that soaring birds can sustain flight for very long distances without almost flapping their wings. This phenomenon has intrigued researchers across multiple disciplines including biology, physics, and applied mathematics. For aerospace and control engineering researchers, mimicking dynamic soaring means new technologies that contribute to a more sustainable aviation industry. Significant work has been done in the literature to mimic dynamic soaring using optimal control frameworks. However, these approaches have limitations as they are non-real-time, model-dependent, and computationally expensive. Very recently, the authors of this paper introduced a novel autonomous, real-time, and model-free approach for mimicking dynamic soaring utilizing Extremum Seeking Control (ESC) methods. However, the ESC structures used in said emerging approach are sensitive to the curvature of the input-output map of the system. Therefore, in this paper, we propose a Newton-based ESC structure for dynamic soaring that is independent of the input-output map's curvature. This provides a twofold contribution: (1) further solidification that the dynamic soaring problem can be treated as a natural ESC system; and (2) a framework that captures dynamic soaring independent of the input-output map's curvature, which can be particularly useful in cases where the system model is unknown. We verify our real-time results via simulations and comparison with non-real-time powerful optimal control solvers.

Control-affine Extremum Seeking Control (ESC) systems have been increasingly studied and applied in the last decade. In a recent effort, many control-affine ESC structures have been generalized in a unifying class and their stability was analyzed. However, guaranteeing vanishing oscillations at the extremum point for said class requires strong conditions that may not be feasible or easy to check/design by the user, especially when the gradient of the objective function is unknown. In this paper, we introduce a control-affine ESC structure that remedies this problem such that: (i) its oscillations attenuate structurally via a novel application of a geometric-based Kalman filter and a Lie bracket estimation approach; and (ii) its stability is characterized by a time-dependent (one-bound) condition that is easier to check and relaxed when compared to the generalized approach mentioned earlier. We provide numerical simulations of three problems to demonstrate the effectiveness of our proposed ESC; these problems cannot be solved with vanishing oscillations using the generalized approach in the literature.

Extremum seeking control (ESC) is an adaptive control technique, introduced nearly a century ago, to drive a dynamic system to the extremum of an objective function that may not be known expression-wise. A rigorous stability analysis of the so-called classical ESC structure, using averaging and singular perturbation theory, has increased research topics and applications involving ESC. Another class of ESCs, control-affine in nature and analyzed using Lie bracket system-based approaches, has emerged, but with some limited theoretical advancements compared to classical ESC. Gradient estimation tools are not well-established for such control-affine ESC structures. Also, stability analysis can be challenging due to complex bounds and conditions. So, in this paper, we introduce a geometric-based extended Kalman filter (GEKF) for gradient and Lie bracket estimation in control-affine ESC systems. We also propose a time-dependent stability condition for control-affine ESC based on the Lie bracket system’s evolution with time. This enables real-time stability tracking. The potential and advantage of our results are demonstrated through numerical simulations of two ESC cases in the literature, including a multi-agent problem.

View Video Presentation: https://doi.org/10.2514/6.2023-2239.vid In this paper, we will show that Extremum Seeking Control (ESC) systems, which are autonomous, real-time, and stable are a natural characterization of the Dynamic Soaring (DS) problem. We will formulate the DS problem with ESC as a controller, and perform simulations to obtain the trajectories of UAVs/birds performing the DS maneuver. Then, we will compare them with the trajectories obtained from an optimal control solver, GPOPS2. The results show that the trajectory obtained using ESC is comparable to the optimal trajectory obtained using GPOPS2. Furthermore, we observe that total energy is near-constant in both methods. This shows that the system is conservative and supports our claim that ESC emulates DS. Our proposed method provides an alternative to the control works in the literature associated with DS which rely heavily on constrained optimal control algorithms, control designs that require a mathematical expression of the objective function, and predefined wind profile models. We believe this work will take us closer to the implementation of DS in real applications that would make Unmanned Aerial Vehicles (UAVs) very energy-efficient during part (or the full) duration of their flight.

The ability of an albatross to travel very long distances without any source of power, has drawn researchers' attention. Albatross exploits an inspiring environmental phenomenon named by Dynamic Soaring (DS), through which it extracts energy from the wind -- particularly, when the wind is changing with altitude causing what is known as "wind shear". This paper provides the DS problem setup to be solved as an optimal control problem as typically done in literature. Moreover, We investigate a novel deep reinforcement learning (RL) approach in continuous space to find the optimal control signals corresponding to optimal DS trajectories of bio-mimicking UAVs performing DS and flying in dynamic shear wind conditions. In addition, we also compare the performance of different deep RL algorithms for trajectory optimization in terms of total average reward to identify some candidate approaches. Additionally, we compare the RL results with two numerical optimizers: GPOPS2, which can be applied using Matlab; and a direct collocation based algorithm, which can be applied using Python. We provide discussions and assessments following said comparisons between the newly developed RL framework and the two numerical optimizers above-mentioned.

. The albatross optimized flight maneuver – known as Dynamic Soaring (DS) – is nothing but a wonder of physics, biology, and engineering. In an ideal DS cycle, this fascinating bird can travel in the desired flight direction, for free, by harvesting energy from the wind, and hence, it achieves a neutral energy cycle. This phenomenon has triggered a momentous interest among aeronautical, control and robotic engineering communities; if DS is mimicked, we have arrived at a new class of Unmanned Aerial Vehicles (UAVs) which are very energy-efficient during part (or the full) duration of their flight. However, the DS problem is highly nonlinear, under-actuated, and dependent on the wind profiles. This has resulted in decades of DS control literature that, while making progress in addressing the control problem, seem not to be aligned well with the nature of the DS phenomenon itself. The control works associated with DS in the literature rely heavily on constrained optimal control algorithms, control designs that require a mathematical expression of the objective function, and predefined wind profile models. Clearly, a functioning controller for DS that allows meaningful bio-mimicry of the albatross, needs to be autonomous, real-time, stable, and capable of tolerating the absence of the expression of the objective function (similar to what the bird does). The qualifications of such controller are the very same characteristics of Extremum Seeking Control (ESC) systems. In this paper, we show that ESC systems existing in control literature for decades are a natural characterization of the DS problem. We provide the DS problem setup, design, stability, and simulation results of the introduced ESC systems. The results, supported by comparison with optimal control solvers, emphasize that the DS phenomenon is a natural expression of ESC systems in nature and that DS can be performed autonomously and in real-time with stability guarantees.

In this paper, we develop a fuzzy Proportional-Integral-Derivative (PID) controller for the pitch control of a type-3 DFIG wind turbine. We obtain the fuzzy PID controller by using the Genetic Fuzzy System (GFS) to tune the parameters of the PID controller. The controller regulates the pitch angle of the wind turbine and minimizes fluctuation in power generation in the presence of wind gusts. The number of inputs for the controller is taken as seven and a cascaded fuzzy tree structure is utilized for computational efficiency. Genetic Algorithm (GA) is used to find the optimal values of parameters and rules related to all the Fuzzy Inference Systems (FISs). The training is done in various step input wind scenarios with Integral Square Error (ISE) as the fitness function. The results show that fuzzy the PID controller is better than the PID controller in every test scenario.

The albatross optimized flight maneuver-known as dynamic soaring-is nothing but a wonder of biology, physics, and engineering. By utilizing dynamic soaring, this fascinating bird can travel in the desired flight direction almost for free by harvesting energy from the wind. This phenomenon has been observed for centuries as evidenced by the writings of Leonardo da Vinci and Lord Rayleigh. Moreover, dynamic soaring biological inspiration has triggered a momentous interest among many communities of science and engineering, particularly aeronautical, control, and robotic engineering communities. That is, if dynamic soaring is mimicked, we will have arrived at a new class of unmanned aerial vehicles that are very energy-efficient during part (or the full) duration of their flight. Studying, modeling, and simulating dynamic soaring have been conducted in literature by mostly configuring dynamic soaring as an optimal control problem. Said configuration requires accurate dynamic system modeling of the albatross/mimicking-object, accurate wind profile models, and a defined mathematical formula of an objective function that aims at conserving energy and minimizing its dissipation; the solution then of such optimal control problem is the dynamic soaring trajectory taken-or to be taken-by the bird/mimicking-object. Furthermore, the decades-long optimal control configuration of the dynamic soaring problem resulted in non-real-time algorithms and control solutions, which may not be aligned well with the biological phenomenon itself; experimental observations of albatrosses indicate their ability to conduct dynamic soaring in real-time. Indeed, a functioning modeling and control framework for dynamic soaring that allows for a meaningful bio-mimicry of the albatross needs to be autonomous, real-time, stable, and capable of tolerating the absence of mathematical expressions of the wind profiles and the objective function-hypothetically similar to what the bird does. The qualifications of such modeling and control framework are the very same characteristics of the so-called extremum seeking systems. In this paper, we show that extremum seeking systems existing in control literature for decades are a natural characterization of the dynamic soaring problem. We propose an extremum seeking modeling and control framework for the dynamic soaring problem hypothesizing that the introduced framework captures more features of the biological phenomenon itself and allows for possible bio-mimicking of it. We provide and discuss the problem setup, design, and stability of the introduced framework. Our results, supported by simulations and comparison with optimal control methods of the literature, provide a proof of concept that the dynamic soaring phenomenon can be a natural expression of extremum seeking. Hence, dynamic soaring has the potential to be performed autonomously and in real-time with stability guarantees.

In this paper, we present an algorithm to optimally track a non-cooperative quadrotor using a team of Unmanned Ground Vehicles (UGVs) which are equipped with GPS as well as radar capable of sensing range, azimuth angle and elevation angle measurements. First, a distributed information filter algorithm is developed for each UGV to locally estimate the position and velocity of the quadrotor using its information and information from neighboring UGVs. We then fuse the information from various UGVs using a Consensus Filter such that their estimate of the state of the quadrotor converges. However, the accuracy of the estimation by the consensus filter depends upon the relative position of the sensors. Hence, it is necessary to design a control policy that leads to optimum configuration of UGVs in order to maximize the utility of measurement. A Model Predictive Controller(MPC) is developed to optimally control the UGVs to improve the state estimation. By defining a proper cost function and optimization algorithm in MPC, we are able to control the path of UGVs to increase the accuracy of estimating the position and velocity of a non-cooperative quadrotor.

In the last decade, a control-affine Extremum Seeking Control (ESC) approach was introduced, and mainly applied on multi-agent systems. This ESC structure is compatible with single-integrator and unicycle dynamics to steer collaborative multi-agent units (such as, but not limited to vehicles) to the extremum of an objective function that we have access to its measurements, but not its expression. This approach utilized a Lie bracket approximation-system for stability characterization, however, it possesses persistent oscillations at all time. In the following years, Lie bracket-based approaches have been developed, and eventually, in a recent effort, generalized in a unifying class that under strong conditions converge asymptotically to the extremum point; nevertheless, in this class the extremum point has to be known a priori and guaranteeing vanishing control input at the extremum point requires the application of a strong condition. In this paper, we introduce a Lie bracket-based approach for control-affine ESCs, which is also compatible with multi-agent systems. The proposed ESC does not require the extremum point a priori, its oscillations attenuate structurally, and its stability is characterized by a time-dependent condition that does not require strong bounds or knowledge on the objective function compared with literature. Moreover, we show that the proposed ESC works with some control-affine cases in which the said generalized class could not. We provide a multi-agent vehicle problem to demonstrate our results numerically in direct comparison with literature.

In this paper, we propose a class of Extremum Seeking Controls (ESC) that are compatible with classic structures of ESC, especially those applied to multi-agent systems, among others. The proposed class mixes the philosophy of two main approaches studied in the recent literature of ESC, namely Lie bracket approximation to ESC and the utilization of adaptation laws for the amplitude of the excitation signals. Uniqueness of solutions and stability results are analyzed for the provided class. Moreover, asymptotic convergence of the amplitude of the excitation signal to a user-defined attenuation (including vanishing) is proved under a time-dependent condition that is more relaxed when compared to many similar conditions in the literature. Finally, we provide numerical simulations for a multi-agent system to support our results.

In the last decade, a control-affine Extremum Seeking Control (ESC) approach was introduced, and mainly applied on multiagent systems. This ESC structure is compatible with single-integrator and unicycle dynamics to steer collaborative multi-agents (such as, but not limited to vehicles) to the extremum of an objective function that we have access to its measurements, but not its expression. This approach utilized a Lie bracket approximation for stability characterization, however, it possesses persistent oscillations. In the following years, Lie bracket-based approaches have been developed, and eventually in a recent effort, generalized in a unifying class, which under strong conditions converge asymptotically to the extremum point; nevertheless, in this class the extremum point has to be known a priori. In this paper, we introduce a Lie bracket-based approach for controlaffine ESCs, mainly compatible with multi-agent systems, and differs in its stability characteristics from the said generalized class. The proposed ESC does not require the extremum point a priori, its oscillations attenuate progressively, and its stability is characterized by a simpler time-dependent condition that does not require strong bounds or knowledge on the objective function. We provide a multi-agent vehicle problem to demonstrate our results numerically in direct comparison with literature.

In this paper, we propose a class of Extremum Seeking Controls (ESC) that are compatible with classic structures of ESC, especially those applied to multi-agent systems, among others. The proposed class mixes the philosophy of two main approaches studied in the recent literature of ESC, namely Lie bracket approximation to ESC and the utilization of adaptation laws for the amplitude of the excitation signals. Uniqueness of solutions and stability results are analyzed for the provided class. Moreover, asymptotic convergence of the amplitude of the excitation signal to a user-defined attenuation (including vanishing) is proved under a time-dependent condition that is more relaxed when compared to many similar conditions in the literature. Finally, we provide numerical simulations for a multi-agent system to support our results.