A constrained adaptive actor-critic (AAC)-based policy iteration controller for vision-based shape servoing (VSS) of a deformable object is presented in this article. A finite point-based representation of the deformable object is used, and the deformation Jacobian matrix is approximated using Fourier-series basis functions. An integral concurrent learning (ICL)-based parameter update law is designed to update the deformation model parameters. The high-dimensional state representation of the object requires a large number of parameters to learn the deformation model, value function, and the optimal policy. To address this, a principal component analysis (PCA)-based dimensionality reduction method is used to compute a low-dimensional representation of the object, which is then used to design the constrained AAC controller. A barrier transform (BT) method is used to transform part of the state into a constrained state, thereby ensuring that the physical constraints are satisfied. The proposed controller is extensively tested in simulations across several desired configurations, and through Monte Carlo runs on an ABB IRB120 robot platform.

In this paper, constrained parameter update laws for adaptive control with convex equality constraint on the parameters are developed, one based on a gradient only update and the other incorporating concurrent learning (CL) update. The update laws are derived by solving a constrained optimization problem with affine equality constraints. This constrained problem is reformulated as an equivalent unconstrained problem in a new variable, thereby eliminating the equality constraints. The resulting update law is integrated with an adaptive trajectory tracking controller, enabling online learning of the unknown system parameters. Lyapunov stability of the closed-loop system with the equality-constrained parameter update law is established. The effectiveness of the proposed equality-constrained adaptive control law is demonstrated through simulations, validating its ability to maintain constraints on the parameter estimates, achieving convergence to the true parameters for CL-based update law, and achieving asymptotic and exponential tracking performance for constrained gradient and constrained CL-based update laws, respectively.

In this paper, a continuous-time adaptive actor-critic reinforcement learning (RL) controller is developed for drift-free uncertain nonlinear systems. Practical examples of such systems are imagebased visual servoing (IBVS) and wheeled mobile robots (WMR), where the system dynamics include a parametric uncertainty in the control effectiveness matrix with no drift term. The uncertainty in the input term poses a challenge when developing a continuous-time RL controller using existing methods. This paper presents an actor-critic/synchronous policy iteration (PI)-based RL controller with a newly derived constrained concurrent learning (CCL)-based parameter update law for estimating the unknown parameters of the linearly parametrized control effectiveness matrix. The parameter update law ensures that the parameters do not converge to zero, avoiding possible loss of stabilization. An infinite-horizon value function minimization objective is achieved by regulating the current states to the desired with nearoptimal control efforts. The proposed controller guarantees closed-loop stability, and simulation results in the presence of noise validate the proposed theory using IBVS and WMR examples

In this paper, a constrained parameter update law is developed within the framework of adaptive control. The update law is derived using a constrained optimization approach, where a Lagrangian is formulated to incorporate parameter constraints through an inverse barrier function. This constrained update law is then integrated into the design of an adaptive tracking controller. The overall stability of the closed-loop system, including both the adaptive controller and the constrained update mechanism, is established using Lyapunov analysis and the recent results on the stability of constrained primal-dual dynamics. The effectiveness of the proposed method is demonstrated through simulations, which verify its ability to maintain parameter estimates within prescribed bounds while ensuring convergence to the true parameter values.

In this paper, a constrained parameter update law is derived in the context of adaptive control. The parameter update law is based on constrained optimization technique where a Lagrangian is formulated to incorporate the constraints on the parameters using inverse Barrier function. The constrained parameter update law is used to develop a adaptive tracking controller and the overall stability of the adaptive controller along with the constrained parameter update law is shown using Lyapunov analysis and development in stability of constrained primal-dual dynamics. The performance of the constrained parameter update law is tested in simulation for keeping the parameters within constraints and convergence to true parameters.

This article addresses the stability of switching between a uniformly ultimately bounded (UUB) system and an asymptotically stable system with asymptotically decaying perturbation using multiple Lyapunov functions. It is proven that the switched system trajectories remain UUB if an average dwell time condition is satisfied, and the perturbation terms are bounded with a sufficiently small magnitude. The developed switched system stability results are applied to the state estimation of the perspective dynamical system in the presence of intermittent and biased velocity measurements using switched observers. Numerical simulations demonstrate the advantages of using the developed switched observer versus the individual observers.

The chapter presents two methods of constrained learning of dynamical system models from data with applications to learning from demonstration (LfD) or imitation learning in the context of manufacturing robotics. The advances in the field of LfD is described and important challenges are discussed and supported by related literature. Specifically, the dynamic system (DS) learning-based approach of LfD is described in detail. Two of our recent results of constrained DS model learning are described and their details are provided. The first technical result is a constrained DS model learning method that learns a system model subject to contraction constraints, which provides convergence with respect to a goal state. The second technical result is a constrained learning method to learn a DS model subject to stability and safety constraints. Simulation results are provided. The chapter concludes with some future directions of research.

This paper presents a novel range estimation of moving targets observed by a moving camera. The target motions are modeled using Gaussian Processes (GP). Using GP regression, target velocity models of several basic motions are learned a-priori and stored as a library. An interacting multiple model (IMM) filter is then utilized on the perspective dynamical system (PDS) model to estimate the 3D range of the feature points on the moving target. The IMM selects the most likely target motion model from the bank of motion models such that the measurement likelihood is maximized and the relative range state is estimated from the image observations. Simulation results performed using a target motion that is a combination of basic target motions show good range estimation performance in terms of the root mean square error (RMSE) metric.

In this paper, a continuous-time reinforcement learning (RL)-based controller is developed for image-based visual servoing (IBVS). The IBVS control dynamics is of the form where the drift term is absent and there is an uncertainty in the Jacobian matrix that is multiplied with the input. This poses a challenge for developing a continuous-time RL controller. The paper presents an actor-critic or synchronous policy iteration (PI)-based RL controller along with a parameter update law for the unknown parameter in the image Jacobian and proves closed-loop stability with the proposed controller. An infinite-horizon value function minimization objective is achieved by regulating the current image features to the desired with near-optimal control efforts. The proposed controller is tested using a simulation use case and the results validate the proposed theory.

An adaptive shape servoing control method is presented in this article to manipulate a deformable object into a desired shape in 3-D. A finite-point-based representation of the deformable object is used and the deformation Jacobian matrix is approximated using Fourier series basis functions. The unknown parameters of the deformation Jacobian are learned by using the velocity applied to a control point on the object and corresponding change of positions of the points describing the entire object. An integral concurrent learning (ICL)-based parameter update law is designed along with a constrained controller to satisfy the state constraints on the motion of the control point using Barrier Lyapunov function analysis. ICL-based parameter update law uses data history of velocity and corresponding positions of the points along with their current values. An efficient algorithm to update the history stack using singular value maximization is proposed based on the structure of the regressor matrix. Simulations using a physical simulator and experiments using a robot platform are performed to validate the performance of the proposed controller on two different deformable objects.

This paper presents a novel method for multi-user motion intent estimation when the motion is observed by a single sensor. A motion model is associated with each of the activities carried out by the operator and the end location of which is termed as a motion intent. Such modeling of intent is useful in human-robot collaborative tasks. The appropriate model selection is achieved via an interacting multiple model (IMM) filter. When the position measurements of multiple users originating from one sensor are close to each other, then the measurement to operator association becomes challenging. A joint probabilistic data association (JPDA) filter is employed to address this issue. The combined IMM and JPDA filter provides a way to infer the motion intent of each operator. Simulation results show that the IMM-JPDA filter tracks two target states reaching toward goal intent in the presence of clutter measurements originating from the Kinect sensor.

In this letter, an adaptive trajectory synchronization controller is developed that synchronizes the robot joint trajectory to the human joint trajectory in the presence of communication time delay and uncertainty in robot model parameters including nonlinear-in-parameter friction term. The controller synchronizes to the human trajectory by accounting for time delays that arise in human-robot collaboration tasks such as, estimating the human trajectory using image processing, or sensor fusion for trajectory intent estimation, or computational limitations. The developed adaptive time-delayed synchronization controller utilizes a new integral concurrent learning (ICL)-based parameter update law for Neural Network parameter estimation. Uniformly ultimately bounded stability of the synchronization and parameter estimation errors are proved using a Lyapunov-Krasovskii functional analysis. Results of the Monte Carlo simulations are presented to validate the performance of the proposed synchronization controller using a human-robot synchronization example.

Utilizing perception for feedback control in combination with dynamic movement primitive (DMP)-based motion generation for a robot's end-effector control is a useful solution for many robotic manufacturing tasks. For instance, while performing an insertion task when the hole or the recipient part is not visible in the eye-in-hand camera, a learning-based movement primitive method can be used to generate the end-effector path. Once the recipient part is in the field of view (FOV), image-based visual servo (IBVS) can be used to control the motion of the robot. Inspired by such applications, this article presents a generalized control scheme that switches between motion generation using DMP and IBVS control. To facilitate the design, a common state-space representation for the DMP and the IBVS systems is first established. The stability analysis of the switched system using multiple Lyapunov functions shows that the state trajectories converge to a bound asymptotically. The developed method is validated by three real-world experiments using the eye-in-hand configuration of a Baxter research robot.

This paper presents a discrete-time dynamical system model learning method from demonstration while providing probabilistic guarantees on the safety and stability of the learned model.The controlled dynamic model of a discrete-time system with a zero-mean Gaussian process noise is approximated using an Extreme Learning Machine (ELM) whose parameters are learned subject to chance constraints derived using a discrete-time control barrier function and discrete-time control Lyapunov function in the presence of the ELM reconstruction error.To estimate the ELM parameters a quadratically constrained quadratic program (QCQP) is developed subject to the constraints that are only required to be evaluated at sampled points.Simulations validate that the system model learned using the proposed method can reproduce the demonstrations inside a prescribed safe set while converging to the desired goal location starting from various different initial conditions inside the safe set.Furthermore, it is shown that the learned model can adapt to changes in goal location during reproductions without violating the stability and safety constraints.

In this paper, a data-driven modeling and control framework is developed for task space control of a soft robot gripper which consists of four individual soft fingers. Each of the four fingers is modeled as a manipulator with high degrees of freedom. The corresponding task space dynamics of the manipulator are derived using a rigid-link approximation of the continuum manipulator. A neural network approach is used to learn the derived dynamics in State Dependent Coefficient (SDC) form. Using the learned SDC matrices, an asymptotically stable optimal closed-loop tracking controller which is based on solving the State Dependent Riccati Equation (SDRE) is derived. The model learning and trajectory tracking controller is implemented on an open source Soft Motion (SoMo) platform simulating the soft gripper motion and corresponding tracking results are presented.

The paper presents a robust parameter learning methodology for identification of nonlinear dynamical system from data while satisfying safety and stability constraints in the context of learning from demonstration (LfD) methods. Extreme Learning Machines (ELM) is used to approximate the system model, whose parameters are learned subject to the safety and stability constraints obtained using zeroing barrier and Lyapunov-based stability analysis in the presence of model uncertainties and external disturbances. A constrained Quadratic Program (QP) is developed, which accounts for the ELM function reconstruction error, to estimate the ELM parameters. Furthermore, a robustness lemma is presented, which proves that the learned system model guarantees safety and stability in the presence of disturbances. The method is tested in simulations. Trajectory reconstruction accuracy of the method is compared against state-of-the-art LfD methods using swept error area (SEA) metric. Robustness of the learned model is tested by conducting Monte Carlo tests. The proposed method is implemented on a Baxter robot for a pick-and-place task where the robot is constrained to an ellipsoidal safety region.

EDITORIAL article Front. Robot. AI, 01 July 2022Sec. Robotic Control Systems https://doi.org/10.3389/frobt.2022.949214

In this paper, a deep learning-based multiple model estimation framework is presented for the state estimation of hybrid dynamical systems from high dimensional observations such as camera images. A low dimensional vector which represents the measurement of the latent dynamical system and its corresponding variance are learned using a deep encoder neural network. An Interacting Multiple Model (IMM) filter is used to generate the latent state estimates and covariances using multiple dynamical models, which can be learned using backpropagation through time. The state estimates of the dynamical system and the corresponding covariance matrix are generated from the latent state estimates and covariance using a deep decoder neural network. The whole network is trained in an end-to-end manner using a loss function which minimizes the negative log-likelihood of the neural network parameters. Simulation results are presented using a 2D bouncing ball example and estimation error statistics are computed which demonstrates the accuracy and consistency of the estimation.

This paper presents a discrete-time nonlinear system identification method while satisfying the stability and safety properties of the system with high probability. An Extreme Learning Machine (ELM) is used with a Gaussian assumption on the function reconstruction error. A quadratically constrained quadratic program (QCQP) is developed with probabilistic safety and stability constraints that are only required to be satisfied at sampled points inside the invariant region. The proposed method is validated using two simulation examples: a two degrees-of-freedom (DoF) robot manipulator with constraints on joint angles whose trajectories are guaranteed to remain inside a safe set and on motion trajectories data of a hand-drawn shape.