Motion planning algorithms compute control sequences that drive autonomous robots to goal regions while avoiding unsafe states. Existing methods, from sampling-based planning to deep reinforcement learning, typically provide task-completion guarantees only with respect to a nominal model or simulator, which may be invalidated when the true dynamics are unknown or difficult to model accurately. This letter addresses this limitation for systems with unknown dynamics and an available approximate nominal model, contributing a planner-agnostic constraint-tightening procedure that equips existing planners with a probabilistic task-completion guarantee on the true system. We leverage conformal prediction to provide a probabilistic bound on the nominal-to-true trajectory deviation over a distribution of planning problems. We tighten the planning constraints using that bound, and show that solving the tightened problem under the nominal model is a sufficient condition for solving the original problem on the true system with a prescribed probability. We validate the theoretical guarantees empirically and demonstrate substantially improved task completion relative to nominal-model planning.

Deep Convolutional Neural Networks (CNNs) achieve remarkable performance but lack interpretability, limiting their adoption in high-stakes applications. Existing Class Activation Mapping (CAM) methods suffer from gradient noise sensitivity or computational inefficiency, while lacking principled strategies for selecting informative, non-redundant features. We introduce Gram-Schmidt Feature Selection CAM (GFS-CAM), a novel gradient-free approach that leverages Gram-Schmidt orthogonalization to construct disentangled visual explanations. Our method iteratively builds an orthogonal feature basis by selecting features that explain maximum residual variance. Unlike existing methods that generate single heatmaps, GFS-CAM isolates distinct semantic concepts into separate, interpretable maps, providing fine-grained attribution of model decisions. Extensive experiments on ILSVRC2012 and PASCAL VOC 2007 across multiple CNN architectures (VGG-16, Inception-v3, ResNeXt-50, DeiT-Base) demonstrate superior performance. GFS-CAM achieves state-of-the-art localization accuracy on three of four architectures, ranks first in 10 of 16 evaluation categories across two datasets, and attains best or second-best performance on the ROAD faithfulness benchmark across all tested CNNs. Our approach offers a principled solution that combines the robustness of gradient-free methods with computational efficiency, while providing unprecedented interpretability through feature analysis. Our source code is available at https://github.com/kavehsfv/GFS-CAM.

Replicator dynamics have been widely used in evolutionary game theory to model how strategy frequencies evolve over time in large populations. The so-called payoff matrix encodes the pairwise fitness that each strategy obtains when interacting with every other strategy, and it solely determines the replicator dynamics. If the payoff matrix is unknown, we show in this paper that it cannot be inferred from observed strategy frequencies alone — distinct payoff matrices can induce the same replicator dynamics. We thus look for a canonical representative of the payoff matrix in the equivalence class. The main result of the paper is to show that for every polynomial replicator dynamics (i.e., the vector field is a polynomial), there always exists a skew-symmetric, polynomial payoff matrix that can induce the given dynamics.

This paper introduces and solves a structural controllability problem for ensembles of switched linear systems. All individual subsystems in the ensemble are sparse, governed by the same sparsity pattern, and undergo switching at the same sequence of time instants. The controllability of an ensemble system describes the ability to use a common control input to simultaneously steer every individual system. A sparsity pattern is called structurally controllable for pair (k,q) if it admits a controllable ensemble of q individual systems with at most k switches. We derive a necessary and sufficient condition for a sparsity pattern to be structurally controllable for a given (k,q), and characterize when a sparsity pattern admits a finite k that guarantees structural controllability for (k,q) for arbitrary q. Compared with the linear time-invariant ensemble case, this second condition is strictly weaker. We further show that these conditions have natural connections with maximum flow, and hence can be checked by polynomial algorithms. Specifically, the time complexity of deciding structural controllability is O(n^3) and the complexity of computing the smallest number of switches needed is O(n^3 log n), with n the dimension of each individual subsystem.

We introduce Hyperparameter Controller (HyperController), a computationally efficient algorithm for hyperparameter optimization during training of reinforcement learning neural networks. HyperController optimizes hyperparameters quickly while also maintaining improvement of the reinforcement learning neural network, resulting in faster training and deployment. It achieves this by modeling the hyperparameter optimization problem as an unknown Linear Gaussian Dynamical System, which is a system with a state that linearly changes. It then learns an efficient representation of the hyperparameter objective function using the Kalman filter, which is the optimal one-step predictor for a Linear Gaussian Dynamical System. To demonstrate the performance of HyperController, it is applied as a hyperparameter optimizer during training of reinforcement learning neural networks on a variety of OpenAI Gymnasium environments. In four out of the five Gymnasium environments, HyperController achieves highest median reward during evaluation compared to other algorithms. The results exhibit the potential of HyperController for efficient and stable training of reinforcement learning neural networks.

This paper presents a data-integrated framework for learning the dynamics of fractional-order nonlinear systems in both discrete-time and continuous-time settings. The proposed framework consists of two main steps. In the first step, input-output experiments are designed to generate the necessary datasets for learning the system dynamics, including the fractional order, the drift vector field, and the control vector field. In the second step, these datasets, along with the memory-dependent property of fractional-order systems, are used to estimate the system's fractional order. The drift and control vector fields are then reconstructed using orthonormal basis functions. To validate the proposed approach, the algorithm is applied to four benchmark fractional-order systems. The results confirm the effectiveness of the proposed framework in learning the system dynamics accurately. Finally, the same datasets are used to learn equivalent integer-order models. The numerical comparisons demonstrate that fractional-order models better capture long-range dependencies, highlighting the limitations of integer-order representations.

Feature extraction and selection in the presence of nonlinear dependencies among the data is a fundamental challenge in unsupervised learning. We propose using a Gram-Schmidt (GS) type orthogonalization process over function spaces to detect and map out such dependencies. Specifically, by applying the GS process over some family of functions, we construct a series of covariance matrices that can either be used to identify new large-variance directions, or to remove those dependencies from known directions. In the former case, we provide information-theoretic guarantees in terms of entropy reduction. In the latter, we provide precise conditions by which the chosen function family eliminates existing redundancy in the data. Each approach provides both a feature extraction and a feature selection algorithm. Our feature extraction methods are linear, and can be seen as natural generalization of principal component analysis (PCA). We provide experimental results for synthetic and real-world benchmark datasets which show superior performance over state-of-the-art (linear) feature extraction and selection algorithms. Surprisingly, our linear feature extraction algorithms are comparable and often outperform several important nonlinear feature extraction methods such as autoencoders, kernel PCA, and UMAP. Furthermore, one of our feature selection algorithms strictly generalizes a recent Fourier-based feature selection mechanism (Heidari et al., IEEE Transactions on Information Theory, 2022), yet at significantly reduced complexity.

Multi-robot navigation in cluttered environments presents fundamental challenges in balancing reactive collision avoidance with long-range goal achievement. When navigating through narrow passages or confined spaces, deadlocks frequently emerge that prevent agents from reaching their destinations, particularly when Reinforcement Learning (RL) control policies encounter novel configurations out of learning distribution. Existing RL-based approaches suffer from limited generalization capability in unseen environments. We propose a hybrid framework that seamlessly integrates RL-based reactive navigation with on-demand Multi-Agent Path Finding (MAPF) to explicitly resolve topological deadlocks. Our approach integrates a safety layer that monitors agent progress to detect deadlocks and, when detected, triggers a coordination controller for affected agents. The framework constructs globally feasible trajectories via MAPF and regulates waypoint progression to reduce inter-agent conflicts during navigation. Extensive evaluation on dense multi-agent benchmarks shows that our method boosts task completion from marginal to near-universal success, markedly reducing deadlocks and collisions. When integrated with hierarchical task planning, it enables coordinated navigation for heterogeneous robots, demonstrating that coupling reactive RL navigation with selective MAPF intervention yields a robust, zero-shot performance.

We consider a multi-agent system under the control of a central planner (CP). We model the system as a hierarchical multi-agent reinforcement learning (MARL) problem in which a state process is influenced by both the CP’s control and the agents’ actions. Each agent learns a local policy to maximize a local value function, which may or may not be aligned with the CP’s control objective. The CP’s goal is therefore to find a global policy such that the agents learn an equilibrium joint policy that maximizes the CP’s value function. We first show that this problem is equivalent to a Stackelberg game. Given a model, this equivalence can be used to derive game-theoretic properties about the desired Stackelberg equilibrium and can be used to solve for optimal policies directly. If the game model is unknown, then we propose a Monte Carlo (MC)-based reinforcement learning (RL) method based on the hierarchical game structure. We demonstrate that under standard RL assumptions, this method can approximate solutions to the desired Stackelberg game. This procedure is validated in simulations on synthetic games resembling social welfare problems.

This work studies efficient solution methods for cluster-based control policies of transition-independent Markov decision processes (TI-MDPs). We focus on control of multi-agent systems, whereby a central planner (CP) influences agents to select desirable group behavior. The agents are partitioned into disjoint clusters whereby agents in the same cluster receive the same controls but agents in different clusters may receive different controls. Under mild assumptions, this process can be modeled as a TI-MDP where each factor describes the behavior of one cluster. The action space of the TI-MDP becomes exponential with respect to the number of clusters. To efficiently find a policy in this rapidly scaling space, we propose a clustered Bellman operator that optimizes over the action space for one cluster at any evaluation. We present Clustered Value Iteration (CVI), which uses this operator to iteratively perform "round robin" optimization across the clusters. CVI converges exponentially faster than standard value iteration (VI), and can find policies that closely approximate the MDP's true optimal value. A special class of TI-MDPs with separable reward functions are investigated, and it is shown that CVI will find optimal policies on this class of problems. Finally, the optimal clustering assignment problem is explored. The value functions TI-MDPs with submodular reward functions are shown to be submodular functions, so submodular set optimization may be used to find a near optimal clustering assignment. We propose an iterative greedy cluster splitting algorithm, which yields monotonic submodular improvement in value at each iteration. Finally, simulations offer empirical assessment of the proposed methods.

This work studies a multi-agent Markov decision process (MDP) that can undergo agent dropout and the computation of policies for the post-dropout system based on control and sampling of the pre-dropout system. The controller's objective is to find an optimal policy that maximizes the value of the expected system given a priori knowledge of the agents' dropout probabilities. Finding an optimal policy for any specific dropout realization is a special case of this problem. For MDPs with a certain transition independence and reward separability structure, we assume that removing agents from the system forms a new MDP comprised of the remaining agents with new state and action spaces, transition dynamics that marginalize the removed agents, and rewards that are independent of the removed agents. We first show that under these assumptions, the value of the expected post-dropout system can be represented by a single MDP; this "robust MDP" eliminates the need to evaluate all 2^N realizations of the system, where N denotes the number of agents. More significantly, in a model-free context, it is shown that the robust MDP value can be estimated with samples generated by the pre-dropout system, meaning that robust policies can be found before dropout occurs. This fact is used to propose a policy importance sampling (IS) routine that performs policy evaluation for dropout scenarios while controlling the existing system with good pre-dropout policies. The policy IS routine produces value estimates for both the robust MDP and specific post-dropout system realizations and is justified with exponential confidence bounds. Finally, the utility of this approach is verified in simulation, showing how structural properties of agent dropout can help a controller find good post-dropout policies before dropout occurs.

In stochastic multi-armed bandits, a major problem the learner faces is the trade-off between exploration and exploitation. Recently, exploration-free methods—methods that commit to the action predicted to return the highest reward—have been studied from the perspective of linear bandits. In this paper, we introduce a linear bandit setting where the reward is the output of a linear Gaussian dynamical system. Motivated by a problem encountered in hyperparameter optimization for reinforcement learning, where the number of actions is much higher than the number of training iterations, we propose Kalman filter Observability Dependent Exploration (KODE), an exploration-free method that utilizes the Kalman filter predictions to select actions. Our major contribution of this work is our discovery that the performance of the proposed method is dependent on the observability properties of the underlying linear Gaussian dynamical system. We evaluate KODE via two different metrics: regret, which is the cumulative expected difference between the highest possible reward and the reward sampled by KODE, and action alignment, which measures how closely KODE’s chosen action aligns with the linear Gaussian dynamical system’s state variable. To provide intuition on the performance, we prove that KODE implicitly encourages the learner to explore actions depending on the observability of the linear Gaussian dynamical system. This method is compared to several well-known stochastic multi-armed bandit algorithms to validate our theoretical results.

The paper introduces a linear bandit environment where the reward is the output of a known Linear Gaussian Dynamical System (LGDS). In this environment, we address the fundamental challenge of balancing exploration -- gathering information about the environment -- and exploitation -- selecting to the action with the highest predicted reward. We propose two algorithms, Kalman filter Upper Confidence Bound (Kalman-UCB) and Information filter Directed Exploration Action-selection (IDEA). Kalman-UCB uses the principle of optimism in the face of uncertainty. IDEA selects actions that maximize the combination of the predicted reward and a term that quantifies how much an action minimizes the error of the Kalman filter state prediction, which depends on the LGDS property called observability. IDEA is motivated by applications such as hyperparameter optimization in machine learning. A major problem encountered in hyperparameter optimization is the large action spaces, which hinder the performance of methods inspired by principle of optimism in the face of uncertainty as they need to explore each action to lower reward prediction uncertainty. To predict if either Kalman-UCB or IDEA will perform better, a metric based on the LGDS properties is provided. This metric is validated with numerical results across a variety of randomly generated environments.

This paper presents a Distributed Sensor Reconciliation (DSR) methodology based on the decomposition of a centralized steady-state Kalman Filter (KF), which is used to distributively estimate the state of a LTI plant in the presence of unpredictable sensor faults. A scenario where individual local measurements may not ensure system observability, but their collective combination does, is considered. To tackle this challenge, the proposed DSR scheme aims at hiding the faulty sensor measurements in achieving the locally state estimates and in making all of them converge to the true system’s state. This is accomplished by fusing, at each agent site, both local measurements and communicated estimates from a subset of neighboring agents. The estimation scheme leverages sensor redundancy to enhance robustness against sensor faults, thereby mitigating both multiplicative and additive faults. Finally, theoretical guarantees regarding the stability of the scheme are provided and a simulation example is presented.

This article provides a tool for analyzing mechanisms that aim to achieve resilience against stealthy, or undetectable, attacks on cyber-physical systems (CPSs). We consider attackers who are able to corrupt all of the inputs and outputs of the system. To counter such attackers, a response scheme must be implemented that keeps the attacker from corrupting the inputs and outputs of the system for certain periods of time. To aid in the design of such a response scheme, our tool provides sufficient lengths for these periods of time in order to ensure safety with a particular probability. We provide a conservative upper bound on how long the system can remain under stealthy attack before the safety constraints are violated. Furthermore, we show how a detector limits the set of biases an attacker can exert on the system while still remaining stealthy, aiding a system operator in the design of the detector. Our contributions are demonstrated with an illustrative example.

Controllers can be designed to adapt to dynamic changes in the computational capacity that is available for their execution by adjusting their control computations. The concurrent development of such controllers, and the algorithms for run-time scheduling of these controllers, is investigated. It is shown how a mitigative controller, that can compensate for small errors that are made in computing the control signal during one iteration of the control loop by taking corrective action during the subsequent iteration, can be scheduled by a server-based real-time scheduling algorithm to provide both efficient resource-usage and acceptable control performance. This illustrates that concurrent and reciprocal consideration of mutual adaptivity can yield more resource-efficient implementations as well as better controller performance, than would be possible if scheduling and control were each considered separately.

The Command Governor (CG) approach effectively addresses the problem of enforcing constraints on precompensated systems without modifying existing controllers. However, the prediction model dependence limits its use in cost-sensitive parameter identification applications. Inspired by the recent development of several Data-driven Predictive Control (DPC) algorithms and leveraging behavioral systems theory, this paper proposes a novel data-driven Command Governor scheme that bypasses explicit modeling and does not rely on a parametric system representation. By means of using an input/output trajectory of the plant and a representation of the controller, the proposed data-driven CG handles explicitly both input and output constraints. The effectiveness of the proposed approach is validated through an illustrative example.

Cyber-physical systems are prevalent in many critical infrastructures and are vulnerable to a variety of attacks from the network. Decentralized control systems are particularly vulnerable since they rely heavily on network communication to achieve their goals. To address network vulnerabilities, we present an event-triggered network connection and communication protocol that minimizes the amount of time agents are connected to the network, reducing the window of opportunity for attacks. This mechanism is a function of only local information and ensures stability for the overall system in attack-free scenarios. Our approach distinguishes itself from current decentralized event-triggered control strategies by considering an undirected and connected communication graph and by not assuming that agents are always connected to the network to receive critical information from other agents. An algorithm describing this network connection and communication protocol is provided, and our approach is illustrated via simulation.

Decision-making under uncertainty is a fundamental problem encountered frequently and can be formulated as a stochastic multi-armed bandit problem. In the problem, the learner interacts with an environment by choosing an action at each round, where a round is an instance of an interaction. In response, the environment reveals a reward, which is sampled from a stochastic process, to the learner. The goal of the learner is to maximize cumulative reward. In this work, we assume that the rewards are the inner product of an action vector and a state vector generated by a linear Gaussian dynamical system. To predict the reward for each action, we propose a method that takes a linear combination of previously observed rewards for predicting each action's next reward. We show that, regardless of the sequence of previous actions chosen, the reward sampled for any previously chosen action can be used for predicting another action's future reward, i.e. the reward sampled for action 1 at round t - 1 can be used for predicting the reward for action 2 at round t. This is accomplished by designing a modified Kalman filter with a matrix representation that can be learned for reward prediction. Numerical evaluations are carried out on a set of linear Gaussian dynamical systems and are compared with 2 other well-known stochastic multi-armed bandit algorithms.