Robotic manipulation of objects in cluttered dynamic scenes is challenging for a twofold reason. Object detection and localization are complex due to partial occlusions and high variability in the object classes and manipulation in tight spaces is difficult due to potential collisions. The present letter focuses on the low-level control of the non-prehensile pushing action aimed at moving planar objects of generic shape along a given path with an assigned time law. Based on the continuous and nonlinear dynamics of the system, we propose a nonlinear model predictive controller (NMPC), which avoids the need for linearization and, thus, the hybrid dynamics arising from it. An extensive comparison with a state-of-the-art linear MPC demonstrates that the NMPC can successfully react to more general disturbances, outperforming the linear one. Experimental results confirm the effectiveness of the method in a task where a robot is required to grasp fruits in a container with other obstructing objects (shown in the attached video).

This paper presents an experimental comparison between two existing methods representative of two categories of 6D pose estimation algorithms nowadays commonly used in the robotics community. The first category includes purely deep learning methods, while the second one includes hybrid approaches combining learning pipelines and geometric reasoning. The hybrid method considered in this paper is a pipeline of an instance-level deep neural network based on RGB data only and a geometric pose refinement algorithm based on the availability of the depth map and the CAD model of the target object. Such a method can handle objects whose dimensions differ from those of the CAD. The pure learning method considered in this comparison is DenseFusion, a consolidated state-of-the-art pose estimation algorithm selected because it uses the same input data, namely, RGB image and depth map. The comparison is carried out by testing the success rate of fresh food pick-and-place operations. The fruit-picking scenario has been selected for the comparison because it is challenging due to the high variability of object instances in appearance and dimensions. The experiments carried out with apples and limes show that the hybrid method outperforms the pure learning one in terms of accuracy, thus allowing the pick-and-place operation of fruits with a higher success rate. An extensive discussion is also presented to help the robotics community select the category of 6D pose estimation algorithms most suitable to the specific application.

This paper proposes a novel method to refine the 6D pose estimation inferred by an instance-level deep neural network which processes a single RGB image and that has been trained on synthetic images only. The proposed optimization algorithm usefully exploits the depth measurement of a standard RGB-D camera to estimate the dimensions of the considered object, even though the network is trained on a single CAD model of the same object with given dimensions. The improved accuracy in the pose estimation allows a robot to grasp apples of various types and significantly different dimensions successfully; this was not possible using the standard pose estimation algorithm, except for the fruits with dimensions very close to those of the CAD drawing used in the training process. Grasping fresh fruits without damaging each item also demands a suitable grasp force control. A parallel gripper equipped with special force/tactile sensors is thus adopted to achieve safe grasps with the minimum force necessary to lift the fruits without any slippage and any deformation at the same time, with no knowledge of their weight.

Robotic manipulation in cluttered environments is one of the challenges roboticists are currently facing. When the objects to handle are delicate fresh fruits, grasping is even more challenging. Detecting and localizing fruits with the accuracy necessary to grasp them is very difficult due to the large variability in the aspect and dimensions of each item. This paper proposes a solution that exploits a state-of-the-art neural network and a novel enhanced 6D pose estimation method that integrates the depth map with the neural network output. Even with an accurate localization, grasping fruits with a suitable force to avoid slippage and damage at the same time is another challenge. This work solves this issue by resorting to a grasp controller based on tactile sensing. Depending on the specific application scenario, grasping a fruit might be impossible without colliding with other objects or other fruits. Therefore, a non-prehensile manipulation action is here proposed to push items hindering the grasp of a detected fruit. The pushing from an initial location to a target one is performed by a model predictive controller taking into account the unavoidable delay in the perception and computing pipeline of the robotic system. Experiments with real fresh fruits demonstrate that the overall proposed approach allows a robot to successfully grasp apples in various situations.

This paper deals with the problem of full state estimation for vehicles navigating in a three dimensional space. We assume that the vehicle is equipped with an Inertial Measurement Unit (IMU) providing body-frame measurements of the angular velocity, the specific force, and the Earth’s magnetic field. Moreover, we consider available sensors that provide partial or full information about the position of the vehicle. Examples of such sensors are those which provide full position measurements (e.g., GPS), range measurements (e.g., Ultra-Wide Band (UWB) sensors), inertial-frame bearing measurements (e.g., motion capture cameras), or altitude measurements (altimeter). We propose a generic semi-globally exponentially stable nonlinear observer that estimates the position, linear velocity, linear acceleration, and attitude of the vehicle, as well as the gyro bias. We also provide a detailed observability analysis for different types of measurements. Simulation and experimental results are provided to demonstrate the effectiveness of the proposed estimation scheme.

This paper considers the problem of estimating the position, attitude and velocity of a rigid-body in a 3D space by fusing bearing measurements provided by a monocular camera with gyroscopic and accelerometer measurements provided by an Inertial Measurement Unit (IMU). The proposed deterministic observer is accompanied with an observability analysis, that points out the minimum number of image points (bearings) along with their configuration in the inertial frame, under which local exponential stability is guaranteed. The performance of the observer is demonstrated by performing experiments on a test-bed inertial-Visual sensor.

This paper unveils a novel discovery that the full relative pose of a monocular camera moving in a three dimensional space can be estimated exploiting bearing measurements of only 3 unknown source points (together with velocity measurements) without any additional knowledge if the camera translational motion is sufficiently exciting. The epipolar constraint commonly used in Computer Vision algebraic algorithms for the determination of the so-called essential matrix (all of them require at least 5 source points) is here exploited in the design of the proposed Riccati observer for pose estimation. One remarkable feature of this work is the determination of an explicit persistence of excitation condition that guarantees uniform observability and, subsequently, (local) exponential stability of the proposed observer. Convincing simulation results are provided to support the proposed approach.

This paper deals with the problem of full state estimation for vehicles navigating in a three dimensional space. We assume that the vehicle is equipped with an Inertial Measurement Unit (IMU) providing body-frame measurements of the angular velocity, the specific force, and the Earth’s magnetic field. Moreover, we consider available sensors that provide partial or full information about the position of the vehicle. Examples of such sensors are those which provide full position measurements (e.g., GPS), range measurements (e.g., Ultra-Wide Band (UWB) sensors), inertial-frame bearing measurements (e.g., motion capture cameras), or altitude measurements (altimeter). We propose a generic semi-globally exponentially stable nonlinear observer that estimates the position, linear velocity, linear acceleration, and attitude of the vehicle, as well as the gyro bias. We also provide a detailed observability analysis for different types of measurements. Simulation and experimental results are provided to demonstrate the effectiveness of the proposed estimation scheme.

Homographies provide a robust and reliable cue for visual servo control of robots. Some nonlinear observers have been recently developed for the estimation of temporal sequences of homographies associated with rigid-body motion of a camera observing a stationary planar scene. However, these algorithms do not model well time-varying changes in the homography velocity and tend to perform poorly when the camera or the scene moves fast. In this paper, an internal model-based observer posed on SL(3) for homography estimation is proposed allowing for dealing with complex camera-scene trajectories such as circular and sinusoidal motions of the camera and/or the scene. Rigorous proof of local asymptotic stability is established and excellent performance of the proposed observer is justified by experiments using an IMU-Camera prototype observing an oscillating planar target.

This paper deals with the problem of output regulation for systems defined on matrix Lie-Groups. Reference trajectories to be tracked are supposed to be generated by an exosystem, defined on the same Lie-Group of the controlled system, and only partial relative error measurements are supposed to be available. These measurements are assumed to be invariant and associated to a group action on a homogeneous space of the state space. In the spirit of the internal model principle the proposed control structure embeds a copy of the exosystem kinematic. This control problem is motivated by many real applications fields in aerospace, robotics, projective geometry, to name a few, in which systems are defined on matrix Lie-groups and references in the associated homogenous spaces.

This paper addresses the output regulation problem of rigid bodies whose kinematic configuration space lies on the Special Euclidean Group SE(3). Reference trajectories to be tracked are generated by an autonomous system, referred to as exosystem, defined on the Special Euclidean Group as well. Only partial relative pose measurements associated to the “natural” linear left group action on SE(3) along with the pose and the velocity of the controlled body are available. The proposed control action embeds a copy of the exosystem kinematics properly updated by means of relative information error in the same spirit of internal model principle.

For a class of aerial manipulators given by a quadrotor aerial vehicle endowed with a robotic arm, this work focuses on the design of a feedback control strategy to let the position and the orientation of the end-effector to track a desired trajectory. Different configurations of robotic arms are taken into account so as to link the number of actuated degrees of freedom to the achievable tracking performances. In particular, when a robotic arm characterized by the minimum number of actuated joints is considered, it is shown how the presence of an offset between the point in which the manipulator is attached and the center of gravity of the vehicle may introduce some unstable internal dynamics affecting the achievable asymptotic performances. Such a limitation can be removed by properly exploiting robotic manipulators characterized by a redundant number of actuated joints. The proposed feedback control strategy is then shown to be able to exploit the mechanical layout of the manipulator so as to achieve asymptotic or practical tracking of the desired references.