This paper presents a dense depth estimation approach from light-field (LF) images that is able to compensate for strong rolling shutter (RS) effects. Our method estimates RS compensated views and dense RS compensated disparity maps. We present a two-stage method based on a 2D Gaussians Splatting that allows for a “render and compare" strategy with a point cloud formulation. In the first stage, a subset of sub-aperture images is used to estimate an RS agnostic 3D shape that is related to the scene target shape “up to a motion". In the second stage, the deformation of the 3D shape is computed by estimating an admissible camera motion. We demonstrate the effectiveness and advantages of this approach through several experiments conducted for different scenes and types of motions. Due to lack of suitable datasets for evaluation, we also present a new carefully designed synthetic dataset of RS LF images. The source code, trained models and dataset will be made publicly available at: https://github.com/ICB-Vision-AI/DenseRSLF

In this paper, we propose an approach to address the problem of 3D reconstruction of scenes from a single image captured by a light-field camera equipped with a rolling shutter sensor. Our method leverages the 3D information cues present in the light-field and the motion information provided by the rolling shutter effect. We present a generic model for the imaging process of this sensor and a two-stage algorithm that minimizes the re-projection error while considering the position and motion of the camera in a motion-shape bundle adjustment estimation strategy. Thereby, we provide an instantaneous 3D shape-and-pose-and-velocity sensing paradigm. To the best of our knowledge, this is the first study to leverage this type of sensor for this purpose. We also present a new benchmark dataset composed of different light-fields showing rolling shutter effects, which can be used as a common base to improve the evaluation and tracking the progress in the field. We demonstrate the effectiveness and advantages of our approach through several experiments conducted for different scenes and types of motions. The source code and dataset are publicly available at: https://github.com/ICB-Vision-AI/RSLF

This letter formulates a generic representation of a path-following controller operating under contained motion, which was developed in the context of surgical robotics. It reports two types of constrained motion: i) Bilateral Constrained Motion, also called Remote Centre Motion (RCM), and ii) Unilaterally Constrained Motion (UCM). To deal with the constrained motion of a surgical tool when performing a path-following task inside a cavity, we investigated methods that combine the two tasks operating in a hierarchical manner. The proposed solutions were successfully evaluated, first on our simulator that mimics realistic conditions of middle ear surgery, then on an experimental platform. Different validation scenarios were carried out experimentally to assess quantitatively and qualitatively each developed approach. Although ultimate precision was not the goal of this letter, our concept is validated with enough accuracy $(\leq 100 ~\mu m)$ for the ear surgery.

This article formulates a generic representation of a path-following controller operating under contained motion, which was developed in the context of surgical robotics. It reports two types of constrained motion: i) Bilateral Constrained Motion, also called Remote Center Motion (RCM), and ii) Unilaterally Constrained Motion (UCM). In the first case, the incision hole has almost the same diameter as the robotic tool. In contrast, in the second state, the diameter of the incision orifice is larger than the tool diameter. The second case offers more space where the surgical instrument moves freely without constraints before touching the incision wall. The proposed method combines two tasks that must operate hierarchically: i) respect the RCM or UCM constraints formulated by equality or inequality, respectively, and ii) perform a surgical assignment, e.g., scanning or ablation expressed as a 3D path-following task. The proposed methods and materials were tested first on our simulator that mimics realistic conditions of middle ear surgery, and then on an experimental platform. Different validation scenarios were carried out experimentally to assess quantitatively and qualitatively each developed approach. Although ultimate precision was not the goal of this work, our concept is validated with enough accuracy (inferior to 100 micrometres) for ear surgery.

In this study, we present a new approach to improving vocal fold access to perform phonomicrosurgery. It is done by shooting the laser through a mirror to reach the vocal fold hidden parts. A geometrical study of laser shooting path was conducted for vocal fold anatomical constraints, followed by devising a laser-shooting system conceptual design. Control laws were developed and tested by simulation and validated experimentally on a test bench in a monocular and stereoscopic configuration. Simulation and experimental results are provided to demonstrate the effectiveness of the developed approach.

In this paper, we address the calibration of a family of magnetic manipulation systems composed of several coils that are moved around by serial robot manipulators. We show in this paper that the calibration of the whole system ultimately results in calibrating the manipulator and coil separately up to an unknown rigid transformation. For calibration of the coil, we propose to use a model that has not been used so far in the literature; a control-oriented model which is sufficiently accurate and computes the magnetic field in real time. A protocol for calibrating the magnetic manipulation system using the Nelder–Mead algorithm to estimate the model parameters is presented. Calibration was performed through simulations and validated experimentally on a physical system. It was observed that the root mean square error was reduced by 37% after calibration of the physical system, indicating an improvement in accurately estimating the magnetic model.

In this paper, we propose a new type of continuum robot, referred to as a magnetic concentric tube robot (M-CTR), for performing minimally invasive surgery in narrow and difficult-to-access areas. The robot combines concentric tubes and magnetic actuation to benefit from the ‘follow the leader’ behaviour, the dexterity and stability of existing robots, while targeting millimetre-sized external diameters. These three kinematic properties are assessed through numerical and experimental studies performed on a prototype of a M-CTR. They are performed with general forward and inverse kineto-static models of the robot, continuation and bifurcation analysis, and a specific experimental setup. The prototype presents unique capabilities in terms of deployment and active stability management, while its dexterity in terms of tip orientability is also among the best reported for other robots at its scale.

In this paper, we propose an origami-based auxetic tunable Helmholtz resonator by controlling the volume of the cavity to allow a real-time tunability of the resonance frequency. The design is inspired by the waterbomb origami base, which expands when stretched and contracts when compressed. Such a foldable structure offers a wide range of volume variation, which corresponds to a larger frequency shift in the bandwidth of interest. Based on the design of the origami resonator, multi-physical models are established to understand and predict the parameters' effect on the device's behavior. After fabrication, experimental tests are conducted on physical prototypes to validate the models and illustrate the effectiveness of the concept.

Concentric tube robots (CTRs) have a great potential for use in medical applications. Coupled with a follow-the-leader (FTL) deployment, they allow navigation in constrained environments. However, they are subject to instabilities if one makes use of high curvatures for the tubes, long overlapping lengths of their curved sections, or long transmission lengths. One approach to improve their stability is to pattern the tubes of which they are composed by local removals of material along their lengths. Applying patterns on tubes was proved to be of interest for given deployed lengths of a CTR. In this article, we present a method to enlarge the application field of CTRs that deploy in a follow-the-leader manner, by integrating tube patterning in the design process, with a stability criterion. Our method allows the designer to determine a custom pattern geometry to theoretically ensure the stability of CTRs made of any number of constant curvature tubes, for a complete FTL deployment sequence, and while respecting a desired shape during deployment.

Laser microsurgery is the current gold standard surgical technique for the treatment of selected diseases in delicate organs such as the larynx. However, the operations require large surgical expertise and dexterity, and face significant limitations imposed by available technology, such as the requirement for direct line of sight to the surgical field, restricted access, and direct manual control of the surgical instruments. To change this status quo, the European project μRALP pioneered research towards a complete redesign of current laser microsurgery systems, focusing on the development of robotic micro-technologies to enable endoscopic operations. This has fostered awareness and interest in this field, which presents a unique set of needs, requirements and constraints, leading to research and technological developments beyond μRALP and its research consortium. This paper reviews the achievements and key contributions of such research, providing an overview of the current state of the art in robot-assisted endoscopic laser microsurgery. The primary target application considered is phonomicrosurgery, which is a representative use case involving highly challenging microsurgical techniques for the treatment of glottic diseases. The paper starts by presenting the motivations and rationale for endoscopic laser microsurgery, which leads to the introduction of robotics as an enabling technology for improved surgical field accessibility, visualization and management. Then, research goals, achievements, and current state of different technologies that can build-up to an effective robotic system for endoscopic laser microsurgery are presented. This includes research in micro-robotic laser steering, flexible robotic endoscopes, augmented imaging, assistive surgeon-robot interfaces, and cognitive surgical systems. Innovations in each of these areas are shown to provide sizable progress towards more precise, safer and higher quality endoscopic laser microsurgeries. Yet, major impact is really expected from the full integration of such individual contributions into a complete clinical surgical robotic system, as illustrated in the end of this paper with a description of preliminary cadaver trials conducted with the integrated μRALP system. Overall, the contribution of this paper lays in outlining the current state of the art and open challenges in the area of robot-assisted endoscopic laser microsurgery, which has important clinical applications even beyond laryngology.

This letter deals with the development of a vision based control law to achieve high-accuracy automatic six degrees of freedom (DoF) positioning tasks. The objective of this work is to be able to replace a biological sample under an optical device for a non-invasive depth examination at any given time (i.e., performing repetitive and accurate optical characterizations of the sample). The optical examination, also called optical biopsy, is performed thanks to an optical coherence tomography (OCT) system. The OCT device is used to perform a 3-dimensional optical biopsy, and as a sensor to control the robot motion during the repositioning process. The proposed visual servoing controller uses the 3D pose of the studied biological sample estimated directly from the C-scan OCT images using a Principal Component Analysis (PCA) framework. The proposed materials and methods were experimentally validated using a spectral-domain OCT and a 6-DoF robotic platform. The obtained results have demonstrated the pertinence of such methods which offer a positioning accuracy around 0.052 ± 0.03 mm (mean error ± standard deviation) for linear errors and 0.41 ± 0.16° for angular ones over a 8 × 9 × 3.5 mm 3 workspace.

SUMMARYConstrained motion is essential for varying robotics tasks, especially in surgical robotics, for instance, the case of minimally invasive interventions. This article proposes generic formulations of the classical bilateral constrained motion (i.e., when the incision hole has almost the same diameter as that of the tool) as well as unilaterally constrained motion (i.e., when the hole incision has a larger diameter compared to the tool diameter). One of the latter constraints is combined with another surgical task such as incision/ablation or suturing a wound (modeled here by 3D geometric paths). The developed control methods based on the hierarchical task approach are able to manage simultaneously the constrained motion (depending on the configuration case, i.e., bilateral or unilateral constraint) and a 3D path following. In addition, the proposed methods can operate with both straight or curved surgical tools. The proposed methods were successfully validated in various scenarios. Foremost, a simulation framework was proposed to access the performances of each proposed controller. Thereafter, several experimental validations were carried out. Both the simulation and experimental results have demonstrated the relevance of the proposed approach, as well as promising performances in terms of behavior as well as accuracy.

The use of untethered miniature swimmers is a promising trend, especially in biomedical applications. These swimmers are often operated remotely using a magnetic field commonly generated using fixed coils that can suffer from a lack of compactness and heating issues. The analysis of the swimming capabilities is still an ongoing topic of research. In this letter, we focus on the ability of a magnetic actuation system to operate the propulsion of miniature swimmers with flexible flagellum. As a first contribution, we present a new manipulability criterion to assess the ability of a magnetic actuation system to operate a swimming robot, i.e. to ensure a displacement in any desired direction with a fixed minimum speed. This criterion is developed thanks to an analogy with cable-driven parallel robots. As a second contribution, this manipulability criterion is exploited to identify the dexterous swimming workspace which can be used to design of new coil configurations as well as to highlight the possibilities of moving coil systems. A case study for a planar workspace surrounded by three coils is in particular carried out. The accompanying video illustrates the application of the proposed criterion in 3D, for a large number of coils.

resonators have been historically one of the first engineered concepts for acoustic control. They can be used either for increasing sound at a specific frequency, or create antiresonances depending on the way they are implemented. In terms of acoustic control for industrial applications, they constitute very efficient solutions for tonal excitations with a fixed frequency. The resonance frequency of the resonator has to be tuned on this frequency, meaning that if any change occurs in the excitation frequency, the control is no longer possible with the resonator. The concept proposed here to overcome this limitation consists in designing a Helmholtz resonator with a variable volume, that can be controlled in real-time. On the basis of waterbomb origami design, multiphysics simulations are performed to define the geometry of the adaptive cavity. After manufacturing, experimental tests are used to validate the models and illustrate the efficiency of the concept.

Tensegrity mechanisms are self-stressed and deployable mechanisms which provide interesting properties such as high resistance-to-mass ratio and compliance. Despite a growing interest for these mechanisms in robotics, the actuation selection of such mechanisms is still poorly discussed. In this paper, the influence of the actuation type and position, i.e. actuation mode, within the structure is assessed for a widely considered tensegrity mechanism, the Snelson cross. Actuation strategies of interest are proposed and performance criteria are defined to achieve fair comparison of the mechanisms. Performance maps are generated and the most interesting results are discussed. Finally, building blocks providing elementary motions are identified and compliant equivalences of conventional joints are proposed for the design of more complex tensegrity-based devices. (C) 2020 Elsevier Ltd. All rights reserved.

This paper focuses on the design of a 6 degrees of freedom visual servoing control law. Instead of using geometric visual features in standard vision-based approaches, the proposed controller makes use of wavelet coefficients as control signal inputs. It uses the multiple resolution coefficients representing the wavelet transform of an image in the spatial domain. The main contributions are the definition of the multiple resolution wavelet interaction model that links the time-variation of wavelet coefficients to the robot spatial velocity and the associated task function controller. The proposed control law was tested and validated experimentally using a commercial micromanipulator in an eye to hand configuration. To be able to judge the efficiency of the control law, several validation tests were carried out under different conditions of use i.e., large illumination variations, noisy images, partial occlusions, and using unknown three-dimensional scenes. It is also demonstrated experimentally that the proposed approach outperforms, the well-known photometry visual servoing as well as a feature-based visual servoing, namely in unfavorable conditions of use.

Concentric tube robot (CTR) is a promising class of continuum robot for medical interventions [1] which require to operate through natural orifices, such as trans-nasal skull base surgery [2], middle-ear surgery [3] and trans-urethral surgery [4]. During these applications, the access to the surgical site is difficult due to the narrowness of the natural orifices. In addition, capability to orient the surgical tool at the surgical site is of importance. CTR consists in a telescopic assembly of pre-curved tubes. The elastic interaction between the tubes creates distributed forces and torques which deform the robot backbone. The rotation and translation of the tubes allow then to control the shape of the robot during its deployment through natural orifices, as well as its tip pose during the operation. The capability of CTR to orient their tip while maintaining the same tip position at the surgical site, that is referred as orientability in [5], is of interest and actually limited in the abovementioned applications. Orientability is indeed not yet a criterion for CTR tube selection, and it is even today still difficult to assess. This work is dedicated to the development of a numerical evaluation of CTR orientability.

Flexible and helical magnetic microswimmers have been well reviewed in the literature because they could be exploited for envisaged applications such as targeted drug delivery, material removal, and micromanipulation. In this article, scaled-up versions of those robots are introduced to study in detail their maneuverability and dexterity while swimming. The robots were immersed in pure glycerol, thus, reproducing a low Reynolds scenario. The proposed robots were previously optimized, achieving their best performances. The experiments assess the performances of these two kinds of robots in terms of rapidity, and steering error following 3D trajectories in environments with high viscous variations.

This paper proposes a new control-oriented model to compute the magnetic field created by a coil. A major challenge for untethered microscale mobile robotics is the control of objects for precise and fast displacements. In this work, we propose to use an alternative implementation of a model based on elliptic integral functions to control magnetically actuated micro-robots. It allows to compute the magnetic field even in the area close to the coil quickly and accurately. This model is evaluated numerically and compared to classical approaches — dipole approximation, map-based interpolation and classical elliptic integral models — in terms of accuracy, computation time and memory requirement. Simulation results show that this works allows to have an accurate model in the whole workspace by avoiding numerical issues encountered in previous works. It can be computed in a few milliseconds, making it the right candidate for closed-loop control of magnetically actuated micro-robots.