The use of machine learning in fluid dynamics is becoming more common to expedite the computation when solving forward and inverse problems of partial differential equations. Yet, a notable challenge with existing convolutional neural network (CNN)-based methods for data fidelity enhancement is their reliance on specific low-fidelity data patterns and distributions during the training phase. In addition, the CNN-based method essentially treats the flow reconstruction task as a computer vision task that prioritizes the element-wise precision which lacks a physical and mathematical explanation. This dependence can dramatically affect the models' effectiveness in real-world scenarios, especially when the low-fidelity input deviates from the training data or contains noise not accounted for during training. The introduction of diffusion models in this context shows promise for improving performance and generalizability. Unlike direct mapping from a specific low-fidelity to a high-fidelity distribution, diffusion models learn to transition from any low-fidelity distribution towards a high-fidelity one. Our proposed model - Physics-informed Residual Diffusion, demonstrates the capability to elevate the quality of data from both standard low-fidelity inputs, to low-fidelity inputs with injected Gaussian noise, and randomly collected samples. By integrating physics-based insights into the objective function, it further refines the accuracy and the fidelity of the inferred high-quality data. Experimental results have shown that our approach can effectively reconstruct high-quality outcomes for two-dimensional turbulent flows from a range of low-fidelity input conditions without requiring retraining.

Collective behavior arises when locally interacting units produce coordinated global organization, from synchronization in dynamical systems to task-relevant information flow on graphs. The central challenge is not only to explain how collective behavior emerges, but to design local interaction rules that can produce desired global organization and generalize across graphs, dynamics and tasks.To address this challenge, we introduce the Swarm-Inspired Emergent Synchronizer (SIES), a graph-dynamical framework that learns generalizable local-interaction laws for controllable collective organization. Each node is an agent-like dynamical unit with a state and task cue, and signed source-target-conditioned attention acts as an adaptive coupling term inside an explicit evolution model. Therefore, SIES combines an explicit dynamical engine with local agent intelligence, similar to biological swarms. For synchronization control, SIES learns a generalizable coupling operator that produces prescribed synchronization patterns for CDSs across untrained network scales, target phase relations, and intrinsic node dynamics without retraining. The learned operator also reaches gait-related modes faster than three oscillator baselines and generalizes synchronization-driven locomotion to simulated multi-legged robots of different scales and a physical hexapod after leg disablement. For graph representation learning, SIES applies the same signed interaction principle to message passing and achieves the highest performance among the compared methods on heterophilous node-classification benchmarks. Together, these results position SIES as a generalizable and learnable graph-dynamical interaction framework with promise for synchronization control, adaptive robot coordination, and heterophilous graph representation learning.

This paper studies smoothed matrix-polar spectral gradient flows for unconstrained matrix-valued optimization.The canonical polar-factor map loses smoothness at rank-deficient matrices and becomes ill-conditioned as singular values approach zero, creating analytical difficulties.We therefore introduce a spectral feedback law generated by a smooth spectral potential and establish the regularity, monotonicity, boundedness, and dissipation properties of the feedback.Based on this feedback law, we propose a smoothed spectral gradient flow and prove well-posedness and global convergence of the flow.We derive convergence-rate results for the spectral gradient flow in nonconvex, convex, and Polyak–Lojasiewicz (PL) settings and analyze the Lyapunov structure and convergence of a momentum-augmented system under the same spectral feedback law. Furthermore, we provide a local descent-rate comparison between the smoothed spectral-gradient direction and the standard Frobenius-gradient direction using a general Hessian-based quadratic model. This analysis yields a verifiable normalized descent-rate advantage condition, showing that the local benefit of the spectral direction depends on both first-order alignment with the gradient matrix and the directional curvature induced by the Hessian.

State Space Models (SSMs) have emerged as a powerful paradigm for efficient long-sequence modeling, offering parallel training and fast linear-time recurrent inference. However, like other recurrent architectures, SSMs must compress an unbounded history into a fixed-size state, which limits context retention and makes precise retrieval over long-range context inherently difficult. To overcome this limitation, we propose Delay State Space Models (DSSMs), a delay differential equation (DDE)-inspired extension of diagonal SSMs that augments discrete SSM recurrences with explicit delayed-state feedback. Making explicit delayed feedback practical requires new stability parameterization, history management, and FFT-training tools. We address these challenges with a practical discretization and parameterization grounded in a simple delay-independent stability condition. To bypass direct time-domain kernel construction, we derive the DSSM transfer function and compute kernels in the frequency domain, using a kernel contour shift to suppress aliasing and recover accurate FFT training. Empirically, DSSMs substantially improve targeted delayed-retrieval tasks while outperforming S4D on most standard sequence metrics and remaining close on the others.

Modern language models are built primarily from Transformers, recurrent models, and their hybrid architectures. Transformers rely on token-level attention memories, while recurrent models such as state space models (SSMs) and linear attention maintain compact recurrent states. These architectures are typically instantiated separately or interleaved at the layer level, leaving open whether a shared memory representation can support both recurrent compression and attention-style retrieval. We study this question through the state space duality (SSD) view of Mamba-2, where the SSM state can be interpreted as a compressed associative key–value (KV) cache. We observe that Mamba-2 decodes token-conditioned values from this state but does not decode token-conditioned keys. Based on this observation, we propose DART (Decoded Attention over Recurrent sTates), which retains the chunk state contributions produced by the Mamba-2 chunked scan as chunk state memories, decodes token-conditioned keys and values from these memories, and performs state-memory attention (SMA) over the resulting KV pairs. The retrieved output is then combined with the native Mamba-2 output through a gated residual connection. DART supports practical training by reusing the Mamba-2 chunked scan and implementing SMA as a FlashAttention-style computation. Our analysis and experiments show that DART substantially reduces the length-dependent inference cache compared with a matched attention baseline (e.g., 75% savings when the chunk size is S=256 and the state size is N=128). Compared with Mamba-2, DART substantially improves associative recall and retrieval while preserving general language-modeling quality.

In decentralized federated learning, the challenges posed by nonconvexity and data heterogeneity frequently result in inefficient training and poor generalization performance. While various optimization algorithms incorporate momentum or gradient tracking (GT) individually, few effectively leverage their synergy to tackle these challenges. To address this, we propose a novel algorithm termed decentralized stochastic gradient descent with double momentum (DSGD-DM). Explicitly differentiating itself from existing approaches, DSGD-DM features a unique combination of the heavy ball method and Nesterov's acceleration within the GT framework. This specific integration not only accelerates the convergence rate but also significantly enhances robustness in handling heterogeneous data distributions. For general nonconvex optimization scenarios, we provide a theoretical proof using the Lyapunov stability method, demonstrating that our algorithm achieves sublinear convergence without relying on restrictive assumptions regarding data heterogeneity. Furthermore, under the Polyak-Lojasiewicz (PL) condition, we show that our algorithm has a convergence rate of O(exp(-tau T-1)Phi(0) +(tau(2)sigma(2)/tau T-2(1))) , where phi(0), tau(1), and tau(2) are positive constants. Empirical evaluations validate that our method significantly outperforms existing approaches in terms of test accuracy, particularly under high levels of data heterogeneity.

ABSTRACT Central pattern generators (CPGs) provide a powerful biological principle for rhythmic coordination, but artificial CPGs often depend on manually designed coupling functions and topology‐specific analysis. This proposal focuses on the generalisation mechanism of a swarm‐inspired central pattern generator (SCPG), in which each oscillator learns local attention‐based interactions with its neighbours and collective synchronisation emerges from these distributed decisions. The central hypothesis is that learnt local attention can serve as a general coupling rule that transfers across network scale and target synchronisation mode. To test this hypothesis, the proposal centres on scale/task generalisation experiments, a simplified phase‐model experiment that probes whether a fixed phase‐dependent coupling rule can generalise across network sizes and ablation studies that identify the architectural source of generalisation. The expected outcome is a focused mechanistic account of why SCPG can replace handcrafted CPG coupling rules with scalable learnt local interaction.

This paper addresses distributed optimization problems with equality constraints within the framework of cooperation-competition network. Motivated by the exceptional performance of proportional-integral-derivative (PID) controllers, we propose an accelerated distributed optimization algorithm to decompose the distributed optimization problem on the cooperation-competition network into two cooperative subnetworks and address them. By leveraging the Lyapunov stability theorem, we establish the exponential convergence of our algorithm over undirected connected and structurally balanced cooperation-competition graphs when the parameters (e.g., kp, ki, kd) are selected within certain ranges, assuming that the local objective functions are smooth and strongly convex. Additionally, we provide guidelines for selecting appropriate parameter values (e.g., kp, ki, kd). Furthermore, we show that the D-PID-CCN has great potential for nonconvex distributed optimization over the cooperation-competition network. Finally, we present the effectiveness and superiority of our proposed algorithms on several numerical simulations.

We consider a distributed online convex optimization problem with inequality constraints, where streaming data is distributed among computing agents over a peer-to-peer network. A key challenge in such settings is the high communication load, which has become a bottleneck for the efficiency of distributed algorithms. To address this issue, we propose a consensus-based primal-dual gradient algorithm with compressed communication (DC-DOPDGD) that employs carefully designed compression operators to significantly reduce the amount of transmitted data, along with an error-compensation technique effectively eliminates compression errors. Both dynamic and static network regret bounds are analyzed for the proposed algorithm, and network constraint violation is used to measure constraint violation. Particularly, we prove that the proposed algorithm achieves a static regret bound of O(T-max{kappa,T-1-kappa}), while the violation of constraints is bounded by O(T1-kappa/2) for convex losses, where T is the time horizon and kappa is an element of(0, 1), which generalize those of centralized algorithms. For strongly convex objectives, our algorithm achieves a static regret bound of O(log(T)) and a constraint violation bound of O(root log(T)T), which also matches that of centralized algorithms and achieves the state of the art in distributed scenarios. Numerical simulations validate our theoretical findings and demonstrate significant reductions in the number of transmitted bits compared to uncompressed distributed counterparts with inequality constraints.

Physics-Informed Neural Networks (PINNs) offer a powerful paradigm for flow reconstruction, seamlessly integrating sparse velocity measurements with the governing Navier-Stokes equations to recover complete velocity and latent pressure fields. However, scaling such models to large spatiotemporal domains is hindered by computational bottlenecks and optimization instabilities. In this work, we propose a robust distributed PINNs framework designed for efficient flow reconstruction via spatiotemporal domain decomposition. A critical challenge in such distributed solvers is pressure indeterminacy, where independent sub-networks drift into inconsistent local pressure baselines. We address this issue through a reference anchor normalization strategy coupled with decoupled asymmetric weighting. By enforcing a unidirectional information flow from designated master ranks where the anchor point lies to neighboring ranks, our approach eliminates gauge freedom and guarantees global pressure uniqueness while preserving temporal continuity. Furthermore, to mitigate the Python interpreter overhead associated with computing high-order physics residuals, we implement a high-performance training pipeline accelerated by CUDA graphs and JIT compilation. Extensive validation on complex flow benchmarks demonstrates that our method achieves near-linear strong scaling and high-fidelity reconstruction, establishing a scalable and physically rigorous pathway for flow reconstruction and understanding of complex hydrodynamics.

This paper introduces a control-theoretic perspective on unconstrained optimization algorithms using the backstepping methods. We model the optimization process as an augmented strict-feedback system given by ẋ_1 = x_2, ẋ_2 = u, and ż = q(x_1,z), with a regulated output y = ∇ f(x_1). This formulation recasts the development of unconstrained optimization algorithms as a feedback control problem, where the goal is to design the input u to ensure y(t) → 0. By employing backstepping, we recursively synthesize the actual feedback law u after initially selecting a virtual control for x_1. For convex objective functions, we develop a general synthesis framework for augmented strict-feedback systems and specialize it to the standard strict-feedback case. This unified framework successfully recovers the constant-parameter Nesterov flow and the proportional-integral-derivative (PID) accelerated optimizer as direct corollaries. We further establish that, given a fixed virtual control, the universal second-step law is inverse optimal with respect to an induced outer-tracking problem. This reveals that the optimality of the control law is conditionally dependent on the target manifold prescribed by the virtual control, rather than holding globally across all possible backstepping designs. Finally, we formulate a formal optimal-backstepping theorem that elevates this optimality principle to the virtual-control stage by solving a reduced Hamilton–Jacobi–Bellman problem. These contributions collectively yield a robust and general backstepping-driven paradigm for the analysis and design of continuous-time unconstrained optimization algorithms.

Accurate state of charge estimation is crucial for ensuring the safe operation, prolonging the lifespan, and optimizing energy management of flow batteries. This paper proposes a state of charge estimation method based on electrochemical impedance spectroscopy and a hybrid neural network of temporal convolutional network and bidirectional long short-term memory network. By integrating temperature, flow rate, frequency, as well as the real and imaginary parts of impedance as input features, the temporal convolutional network component specializes in extracting localized, multi-frequency temporal features from the impedance data, while the bidirectional long short-term memory network module complements this by analyzing time-series information bidirectionally to capture long-term dependencies. Experimental validation demonstrates that the hybrid model achieves high state of charge estimation accuracy, with a mean absolute error of 0.42 %, root mean square error of 0.61 %, and a coefficient of determination of 0.999. Furthermore, under varying temperature and flow rate conditions, the model consistently maintains stable estimation accuracy, with a mean absolute error ranging from 0.94 % to 1.08 % and root mean square error ranging from 1.17 % to 1.58 %. The test results confirm that the proposed method possesses outstanding estimation accuracy, robustness, and generalization capability, providing an effective technical solution for flow battery management systems.

In this paper, we investigate a distributed optimization problem in multi-agent systems, where the cost function is a sum of local cost functions associated with individual agents. Inspired by the outstanding performance of proportional- integral-derivative (PID) controllers in the field of control, we propose the Distributed PID Optimization Algorithm (D-PID) based on output feedback to solve the distributed optimization problem. We aim to establish the exponential convergence of the D-PID algorithm over undirected connected graphs when the local objective functions are smooth and strongly convex. Additionally, we provide guidelines for selecting appropriate parameter values (e.g., $K_{p}, K_{i}$ , and $K_{d}$ ) and analyze the correctness of the algorithm over time-varying interaction graphs. To further reduce unnecessary communication resource consumption, we develop the Distributed PID Optimization Algorithm with Time-Triggered Scheme (D-PID-TT). We theoretically demonstrate that D-PID-TT can converge to an optimal solution at a global exponential convergence rate under the same conditions as D-PID. We also provide guidelines for parameter selection and specify the communication period. Furthermore, we show that the D-PID has great potential for nonconvex distributed optimization. Finally, we present numerical simulations to verify the effectiveness and superiority of our proposed algorithms.

Neural Ordinary Differential Equations (NODEs) area novel family of infinite-depth neural-net models through solving ODEs and their adjoint equations. In this paper, we present a strategy to enhance the training and inference of NODEs by integrating a Proportional-Integral-Derivative (PID) controller into the framework of Heavy Ball NODE, resulting in the proposed PIDNODEs and its generalized version, GPIDNODEs. By leveraging the advantages of control, PIDNODEs and GPIDNODEs can address the stiff ODE challenges by adjusting the parameters (i.e., K-p, K-i and K-d) in the PID module. The experiments confirm the superiority of PIDNODEs/GPIDNODEs over other NODE baselines on different computer vision and pattern recognition tasks, including image classification, point cloud separation and learning long-term dependencies from irregular time-series data for a physical dynamic system. These experiments demonstrate that the proposed models have higher accuracy and fewer function evaluations while alleviating the dilemma of exploding and vanishing gradients, particularly when learning long-term dependencies from a large amount of data.

This paper delves into the investigation of a distributed aggregative optimization problem within a network. In this scenario, each agent possesses its own local cost function, which relies not only on the local state variable but also on an aggregated function of state variables from all agents. To expedite the optimization process, we amalgamate the heavy ball and Nesterov's accelerated method with distributed aggregative gradient tracking, resulting in the proposal of two innovative algorithms, aimed at resolving the distributed aggregative optimization problem. Our analysis demonstrates that the proposed algorithms can converge to an optimal solution at a global linear convergence rate when the objective function is strongly convex with the Lipschitz-continuous gradient, and when the parameters (e.g., step size and momentum coefficients) are chosen within specific ranges. Additionally, we present several numerical experiments to verify the effectiveness, robustness and superiority of our proposed algorithms.

High-performance optimization algorithms are essential in deep learning. However, understanding the behavior of optimization (i.e., learning process) remains challenging due to the instability and weak interpretability of the algorithms. Since gradient-based optimizations can be interpreted as continuous-time dynamical systems, applying feedback control to the dynamical systems that model the optimizers may provide another perspective for exploring more robust, accurate and explainable optimization algorithms. In this study, we present a framework for optimization called controlled heavy-ball optimizer. By employing the proportional-integral-derivative (PID) controller in the optimizer, we develop a deterministic continuous-time optimizer called Proportional-Integral-Derivative Accelerated Optimizer (PIDAO), and provide theoretical convergence analysis of PIDAO in unconstrained (non-)convex optimizations. As a byproduct, we derive PIDAO-family schemes for training deep neural networks by using specific discretization methods. Compared to classical optimizers, PIDAO can be empirically proven a more aggressive capacity to explore the loss landscape with lower computational costs due to the property of PID controller. Experimental evaluations demonstrate that PIDAO can accelerate the convergence and enhance the accuracy of deep learning, achieving state-of-the-art performance compared with advanced algorithms. By drawing a connection between a closed-loop feedback control and optimization algorithms, the authors propose a framework to gain insights into optimization and learning processes based on feedback control theory. The results can improve theoretical justification and explainability of optimization methods.

This note aims to identify and rectify the flaws found in the proof of Chen et al. (2023), specifically in Lemma 2, Lemma 3, Theorem 1, and Theorem 2. While the conclusions of Lemma 2, Lemma 3, and Theorem 1 remain valid, certain aspects of their proofs are found to be flawed. This note provides modifications to address these flaws. Additionally, the statement and proof of Theorem 2 are shown to be incorrect. A corrected Theorem 2 with proof is given.

In this article, we propose a novel nonlinear observer based on neural networks (NNs), called neural observers, for observation tasks of linear time-invariant (LTI) systems and uncertain nonlinear systems. In particular, the neural observer designed for uncertain systems is inspired by the active disturbance rejection control, which can measure the uncertainty in real time. The stability analysis (e.g., exponential convergence rate) of LTI and uncertain nonlinear systems (involving neural observers) are presented and guaranteed, where it is shown that the observation problems can be solved only using the linear matrix inequalities (LMIs). Also, it is revealed that the observability and controllability of the system matrices are required to demonstrate the existence of solutions for LMIs. Finally, the effectiveness of neural observers is verified in three simulation cases, including the X-29A aircraft model, the nonlinear pendulum, and the four-wheel steering vehicle.

Continuous stirred tank reactor (CSTR) is a common reactor in the chemical industry. The accurate observation of the concentration conversion rate of the mixture and the internal temperature of the reaction vessel is a prerequisite for obtaining the desired mixture. This paper proposes a novel observer based on residual neural networks for CSTR systems. Firstly, the mathematical model of the CSTR reaction is given, as well as a detailed description of the structure and equations of the residual neural networks and the designed observer. Then the matrix method is used for the nonlinear isolation of the residual neural networks and the theory of quadratic constraints for nonlinear activation functions of the neural networks is applied. Thus, the convergence of the proposed observer is analyzed theoretically in detail. Finally, the numerical simulations are implemented to demonstrate that the proposed residual neural network-based observer can quickly and accurately observe the state changes during the CSTR reaction.

This paper investigates the problem of adaptive tracking control for quadcopter in the presence of nonlinear configuration uncertainties. It utilizes a real-time brain-inspired learning control (RBiLC) method to address the challenges posed by nonlinear time-varying uncertain instructions. To address the issue of flight control law reconfiguration caused by unknown changes in the fuselage configuration (e.g., propellers or motors), this paper introduces an online learning-evaluation-optimization reconstruction mechanism based on RBiLC. The proposed adaptive learning controller mitigates the need for extensive human resources and reduces the time required for flight controller design. The Lyapunov-Krasovskii function is introduced as a compensatory measure to address the impact of parameter uncertainty on system stability. Furthermore, this paper proposes a signed sinusoidal function perturbation estimate to guide the direction and magnitude throughout the online learning process. The approach conducts a theoretical stability analysis on a quadcopter vehicle considering uncertainties in UAV dynamics modeling. The results demonstrate that the proposed scheme achieves superior control and faster adaptation, enabling the system to ultimately converge to a compact set within a limited time domain. Finally, software-in-the-loop (SITL) simulations and flight verification results are presented to validate the proposed control strategy