We present a two-stage approach for learning stability-certified neural controllers that achieves a reduction of up to ∼95% in training time compared to the state-of-the-art baseline, which introduced monotonic neural Lyapunov architectures. Our method combines monotonic neural Lyapunov functions with fulfillment priority logic (FPL) to efficiently initialize controllers before formal verification.Traditional approaches for jointly learning controllers and neural Lyapunov functions require computationally expensive mixed-integer linear programming (MILP) or satisfiability modulo theory (SMT) solvers at each training iteration, often taking several hours to converge. We address this bottleneck by leveraging FPL to perform early joint initialization of the controller and Lyapunov networks. Building on the monotonic neural network architecture from the baseline, which guarantees non-negativity and a unique global minimum by construction, our method focuses on efficiently satisfying the remaining property of decreasing along trajectories.Existing works focus on maximizing the region of attraction/convergence of the learned controller. In contrast, leveraging FPL allows us to (1) increase learning efficiency substantially and (2) focus on complementary performance metrics, such as convergence rate and control effort minimization, thereby adding significant specification flexibility.In this paper, we encode an approximate Lyapunov-decrease condition in FPL to pre-train the controller and Lyapunov networks, then apply a MILP-based verification/refinement step. This decouples efficient learning from certificate enforcement and allows the FPL specification to include auxiliary objectives (e.g., convergence rate and control effort), whose influence persists through the final MILP pass. The resulting controllers converge rapidly while admitting formal Lyapunov certificates on standard nonlinear control benchmarks.1

Behavior Cloning (BC) is an effective imitation learning technique and has even been adopted in some safety-critical domains such as autonomous vehicles. BC trains a policy to mimic the behavior of an expert by using a dataset composed of only state-action pairs demonstrated by the expert, without any additional interaction with the environment. However, During deployment, the policy observations may contain measurement errors or adversarial disturbances. Since the observations may deviate from the true states, they can mislead the agent into making sub-optimal actions. In this work, we use a global Lipschitz regularization approach to enhance the robustness of the learned policy network. We then show that the resulting global Lipschitz property provides a robustness certificate to the policy with respect to different bounded norm perturbations. Then, we propose a way to construct a Lipschitz neural network that ensures the policy robustness. We empirically validate our theory across various environments in Gymnasium. Keywords: Robust Reinforcement Learning; Behavior Cloning; Lipschitz Neural Network

While humans can successfully navigate using abstractions, ignoring details that are irrelevant to the task at hand, most of the existing approaches in robotics require detailed environment representations which consume a significant amount of sensing, computing, and storage; these issues become particularly important in resource-constrained settings with limited power budgets. Deep learning methods can learn from prior experience to abstract knowledge from novel environments, and use it to more efficiently execute tasks such as frontier exploration, object search, or scene understanding. We propose BoxMap, a Detection-Transformer-based architecture that takes advantage of the structure of the sensed partial environment to update a topological graph of the environment as a set of semantic entities (rooms and doors) and their relations (connectivity). The predictions from low-level measurements can be leveraged to achieve high-level goals with lower computational costs than methods based on detailed representations. As an example application, we consider a robot equipped with a 2-D laser scanner tasked with exploring a residential building. Our BoxMap representation scales quadratically with the number of rooms (with a small constant), resulting in significant savings over a full geometric map. Moreover, our high-level topological representation results in 30.9 % shorter trajectories in the exploration task with respect to a standard method. Code is available at: bit.ly/3F6w2Yl.

We consider the problem of using multiple agents to harvest data from a collection of sensor nodes (targets) scattered across a two-dimensional environment. These targets transmit their data to the agents that move in the space above them, and our goal is for the agents to collect data from the targets as efficiently as possible while moving to their final destinations. The agents are assumed to have a continuous control action, and we leverage reinforcement learning, specifically Proximal Policy Optimization (PPO) with Lagrangian Penalty (LP), to identify highly effective solutions. Additionally, we enhance the controller's robustness by incorporating regularization at each state to smooth the learned policy. We conduct a series of simulations to demonstrate our approach and validate its performance and robustness.

We consider the problem of using an autonomous agent to persistently monitor a collection of targets distributed in a given environment. We generalize existing work by allowing the agent’s dynamics to vary throughout the environment, leading to a hybrid dynamical system. This introduces an additional layer of complexity towards the planning portion of the problem: we must not only identify in which order to visit the points of interest, but also in which order to traverse the regions. We propose a tailored global path planner and prove that it is not only probabilistically complete, but converges in probability to a time-optimal solution. We then design an offline sequence planner together with an online trajectory optimizer. Simulations validate the results.

This paper considers the problem of localizing a set of nodes in a wireless sensor network where both the node positions and communication model parameters are unknown. We assume that a multi-agent system moves in formation through the environment, taking measurements of the Received Signal Strength, and seek a controller that optimizes a performance metric based on the Fisher Information Matrix. We propose a two-stage formation-based receding horizon approach that alternates between estimating the parameters and determining where to move and how to scale the formation to maximally inform the estimation problem. We apply a Dynamic Programming approach to solve the multi-stage look ahead control problem of the first stage, followed by a Particle Swarm Optimization algorithm to determine the best formation configuration in the second stage. We demonstrate our approach using different formation structures and compare it against multiple baselines.

Dip-pen nanolithography (DPN) represents a versatile approach for depositing nanoscale quantities of fluids onto surfaces. Here, we show that overwriting─or patterning fluids onto previously deposited features─results in patterns with predictable size and composition that are useful for combinatorial materials experiments. We combine fluorescence microscopy and inertial sensing to show that multiple fluid reservoirs can be mixed together to realize combinatorial libraries of nanoscale features with known mass and composition. As an example of the utility of this approach from a materials discovery perspective, we employ this process to study the mechanics and swelling behavior of polyethylene glycol hydrogels with different compositions. Given that this approach allows one to prepare compositional gradients using less than a microgram of material and functionality to evaluate these using the versatility of the atomic force microscope, it has tremendous potential for the discovery and optimization of performance materials for catalysis, mechanics, photonics, and electronics.

In this article, we consider the problem of finding an optimal trajectory of a single agent tasked with harvesting data from multiple mobile sensor nodes in a wireless sensor network. We describe data transmission using a free-space broadcast communication model and formulate an optimal control problem to extract the data from all the nodes in minimal time. In a 1-D mission space, we demonstrate that the optimal motion strategy can be expressed in parametric form. Within this strategy, the agent's motion is determined by a sequence of two modes, alternating between full-speed travel and a motion derived from a parametric convex combination function of sensor velocities. We present two approaches for solving the parametric optimal control problem, a gradient-descent-based scheme and a heuristic search method. The effectiveness of these methods is demonstrated through simulations.

This paper presents an accelerated distributed receding horizon controller for cooperative network estimation problems using multiple autonomous agents. Our approach accelerates decision-making by integrating a novel heuristic-based ranking system, significantly reducing the dependency on computationally expensive Nonlinear Programs (NLPs). The reduction of computational complexity enables real-time responses and scalability to large systems while maintaining high levels of estimation accuracy. To mitigate the small loss of performance, we further introduce a method that aims at generating the best solution within a given computational time constraint by leveraging both the newly introduced ranking scheme and the traditional NLP solutions. Numerical simulations demonstrate competitive performance when benchmarked against data-driven offline policies (e.g., RL), showing that our methods achieve good results while having enhanced flexibility and robustness properties due to their online nature.

In Real-Time, Feedback-Driven Single Particle Tracking methods, measurements of the emission intensity from a labeled, nanometer-scale particle are used in a feedback loop to track the motion of the particle as it moves inside its native environment, including within living cells. In this work, we take advantage of Point Spread Function (PSF) engineering techniques that encode the axial position of the particle into the shape of the PSF in the focal plane to eliminate the need for out-of-focal-plane measurements, reducing the complexity of implementation and decreasing the overall measurement time of the control loop. Specifically, we used the Double Helix PSF (DH-PSF) in which a single fluorescent source gives rise to two lobes in the image plane with the lobes rotating in the plane as the particle moves along the optical axis. We designed simple estimators of the relative error between the particle and the tracker, and a simple proportional feedback controller to regulate that error to zero. We explored the efficacy of the approach through simulation studies, demonstrating the tracking of fast-moving particles (with diffusion coefficients up to 10 μm^2/s) over long time periods (multiple seconds).

Recent advances in MRI reconstruction have demonstrated remarkable success through deep learning-based models. However, most existing methods rely heavily on large-scale, task-specific datasets, making reconstruction in data-limited settings a critical yet underexplored challenge. While regularization by denoising (RED) leverages denoisers as priors for reconstruction, we propose Regularization by Neural Style Transfer (RNST), a novel framework that integrates a neural style transfer (NST) engine with a denoiser to enable magnetic field-transfer reconstruction. RNST generates high-field-quality images from low-field inputs without requiring paired training data, leveraging style priors to address limited-data settings. Our experiment results demonstrate RNST's ability to reconstruct high-quality images across diverse anatomical planes (axial, coronal, sagittal) and noise levels, achieving superior clarity, contrast, and structural fidelity compared to lower-field references. Crucially, RNST maintains robustness even when style and content images lack exact alignment, broadening its applicability in clinical environments where precise reference matches are unavailable. By combining the strengths of NST and denoising, RNST offers a scalable, data-efficient solution for MRI field-transfer reconstruction, demonstrating significant potential for resource-limited settings.

We consider the problem of using an autonomous agent to persistently monitor a collection of dynamic targets distributed in an environment. We generalize existing work by allowing the agent's dynamics to vary throughout the environment, leading to a hybrid dynamical system. This introduces an additional layer of complexity towards the planning portion of the problem: we must not only identify in which order to visit the points of interest, but also in which order to traverse the regions. We design an offline high-level sequence planner together with an online trajectory optimizer realizing the computed visiting sequence. We provide numerical experiments to illustrate the performance of our approach.

Deep learning methods have been widely used in robotic applications, making learning-enabled control design for complex nonlinear systems a promising direction. Although deep reinforcement learning methods have demonstrated impressive empirical performance, they lack the stability guarantees that are important in safety-critical situations. One way to provide these guarantees is to learn Lyapunov certificates alongside control policies. There are three related problems: 1) verify that a given Lyapunov function candidate satisfies the conditions for a given controller on a region, 2) find a valid Lyapunov function and controller on a given region, and 3) find a valid Lyapunov function and a controller such that the region of attraction is as large as possible. Previous work has shown that if the dynamics are piecewise linear, it is possible to solve problems 1) and 2) by solving a Mixed-Integer Linear Program (MILP). In this work, we build upon this method by proposing a Lyapunov neural network that considers monotonicity over half spaces in different directions. We 1) propose a specific choice of Lyapunov function architecture that ensures non-negativity and a unique global minimum by construction, and 2) show that this can be leveraged to find the controller and Lyapunov certificates faster and with a larger valid region by maximizing the size of a square inscribed in a given level set. We apply our method to a 2D inverted pendulum, unicycle path following, a 3-D feedback system, and a 4-D cart pole system, and demonstrate it can shorten the training time by half compared to the baseline, as well as find a larger ROA.

We consider the problem of localizing a set of nodes in a wireless sensor network when both their positions and the parameters of their communication model are unknown. We assume that a single mobile agent moves through the environment, taking measurements of the Received Signal Strength (RSS), and seek a controller that optimizes a performance metric based on the Fisher Information Matrix (FIM). Our approach involves two stages. In the first stage, we apply a discrete-time receding horizon controller that determines a sequence of positions to move that maximize a metric based on the FIM. In the second stage, we formulate an optimal control problem to move the agent to the first position in that sequence, using a neural-network based controller to approximate the solution to the Hamilton-Jacobi-Bellman (HJB) equation and from that define a feedback control policy to execute that move. Measurements are collected along the way and after completing the move, the data are used to estimate the parameters and the process begins again. Through simulations we demonstrate that our approach outperforms a baseline as well three other optimization approaches.

This paper considers the problem of localizing a set of nodes in a wireless sensor network when both their positions and the parameters of the communication model are unknown. We assume that a single agent moves through the environment, taking measurements of the Received Signal Strength (RSS), and seek a controller that optimizes a performance metric based on the Fisher Information Matrix (FIM). We develop a receding horizon (RH) approach that alternates between estimating the parameter values (using a maximum likelihood estimator) and determining where to move so as to maximally inform the estimation problem. The receding horizon controller solves a multi-stage look ahead problem to determine the next control to be applied, executes the move, collects the next measurement, and then re-estimates the parameters before repeating the sequence. We consider both a Dynamic Programming (DP) approach to solving the optimal control problem at each step, and a simplified heuristic based on a pruning algorithm that significantly reduces the computational complexity. We also consider a modified cost function that seeks to balance the information acquired about each of the parameters to ensure the controller does not focus on a single value in its optimization. These approaches are compared against two baselines, one based on a purely random trajectory and one on a greedy control solution. The simulations indicate our RH schemes outperform the baselines, while the pruning algorithm produces significant reductions in computation time with little effect on overall performance.

We investigate the problem of optimally observing a finite set of targets using a mobile agent over an infinite time horizon. The agent is tasked to move in a network-constrained structure to gather information so as to minimize the worst-case uncertainty about the internal states of the targets. To do this, the agent has to decide its sequence of target-visits and the corresponding dwell-times at each visited target. For a given visiting sequence, we prove that in an optimal dwelling time allocation the peak uncertainty is the same among all the targets. This allows us to formulate the optimization of dwelling times as a resource allocation problem and to solve it using a novel efficient algorithm. Next, we optimize the visiting sequence using a greedy exploration process, using heuristics inspired by others developed in the context of the traveling salesman problem. Numerical results are included to illustrate the contributions.

We consider the problem of harvesting data from a set of targets in a wireless sensor network using a collection of mobile agents. The targets lie in a one dimensional mission space and broadcast their data while the agents move overhead. The agents are required to collect all the data and move to terminal locations to offload that data. We use a Hamiltonian analysis to show that the optimal control can be described using a parameterized policy and then develop a gradient descent scheme using infinitesimal perturbation analysis (IPA) to calculate the gradients of the cost function with respect to the control parameters. To avoid collisions between agents, we then apply a Control Lyapunov Function-Control Barrier Function (CLF-CBF) technique to ensure the agents closely track the desired optimal trajectory to complete their mission while avoiding any collisions. Additionally, we analyze the problem of symmetric deadlock in the CLF-CBF controller and show that it can be avoided by adding a small perturbation to the initial and final heights of the agents over the mission space. The approach is demonstrated through simulation.

We consider the problem of harvesting data from a set of targets distributed throughout a two dimensional environment. The targets broadcast their data to an agent flying above them, and the goal is for the agent to extract all the data and move to a desired final position in minimum time. While previous work developed optimal controllers for the one-dimensional version of the problem, such methods do not extend to the 2-D setting. Therefore, we first convert the problem into a Markov Decision Process in discrete time and then apply reinforcement learning to find high performing solutions using double deep Q learning. We use a simple binary cost function that directly captures the desired goal, and we overcome the challenge of the sparse nature of these rewards by incorporating hindsight experience replay. To improve learning efficiency, we also utilize prioritized sampling of the replay buffer. We demonstrate our approach through several simulations, which show a similar performance as an existing optimal controller in the 1-D setting, and explore the effect of both the replay buffer and the prioritized sampling in the 2-D setting.

Real-time feedback-driven single-particle tracking (RT-FD-SPT) is a class of techniques in the field of single-particle tracking that uses feedback control to keep a particle of interest in a detection volume. These methods provide high spatiotemporal resolution on particle dynamics and allow for concurrent spectroscopic measurements. This review article begins with a survey of existing techniques and of applications where RT-FD-SPT has played an important role. We then systematically discuss each of the core components of RT-FD-SPT in order to develop an understanding of the trade-offs that must be made in algorithm design and to create a clear picture of the important differences, advantages, and drawbacks of existing approaches. These components are feedback tracking and control, ranging from simple proportional-integral-derivative control to advanced nonlinear techniques, estimation to determine particle location from the measured data, including both online and offline algorithms, and techniques for calibrating and characterizing different RT-FD-SPT methods. We then introduce a collection of metrics for RT-FD-SPT to help guide experimentalists in selecting a method for their particular application and to help reveal where there are gaps in the techniques that represent opportunities for further development. Finally, we conclude with a discussion on future perspectives in the field.

Deep learning methods are widely used in robotic applications. By learning from prior experience, the robot can abstract knowledge of the environment, and use this knowledge to accomplish different goals, such as object search, frontier exploration, or scene understanding, with a smaller amount of resources than might be needed without that knowledge. Most existing methods typically require a significant amount of sensing, which in turn has significant costs in terms of power consumption for acquisition and processing, and typically focus on models that are tuned for each specific goal, leading to the need to train, store and run each one separately. These issues are particularly important in a resource-constrained setting, such as with small-scale robots or during long-duration missions. We propose a single, multi-task deep learning architecture that takes advantage of the structure of the partial environment to predict different abstractions of the environment (thus reducing the need for rich sensing), and to leverage these predictions to simultaneously achieve different high-level goals (thus sharing computation between goals). As an example application of the proposed architecture, we consider the specific example of a robot equipped with a 2-D laser scanner and an object detector, tasked with searching for an object (such as an exit) in a residential building while constructing a topological map that can be used for future missions. The prior knowledge of the environment is encoded using a U-Net deep network architecture. In this context, our work leads to an object search algorithm that is complete, and that outperforms a more traditional frontier-based approach. The topological map we produce uses scene trees to qualitatively represent the environment as a graph at a fraction of the cost of existing SLAM-based solutions. Our results demonstrate that it is possible to extract multi-task semantic information that is useful for navigation and mapping directly from bare-bone, non-semantic measurements.