This paper develops an autonomous tethered aerial visual assistant for robot operations in unstructured or confined environments. Robotic tele-operation in remote environments is difficult due to lack of sufficient situational awareness, mostly caused by the stationary and limited field-of-view and lack of depth perception from the robot's onboard camera. The emerging state of the practice is to use two robots, a primary and a secondary that acts as a visual assistant to overcome the perceptual limitations of the onboard sensors by providing an external viewpoint. However, problems exist when using a tele-operated visual assistant: extra manpower, manually chosen suboptimal viewpoint, and extra teamwork demand between primary and secondary operators. In this work, we use an autonomous tethered aerial visual assistant to replace the secondary robot and operator, reducing human robot ratio from 2:2 to 1:2. This visual assistant is able to autonomously navigate through unstructured or confined spaces in a risk-aware manner, while continuously maintaining good viewpoint quality to increase the primary operator's situational awareness. With the proposed co-robots team, tele-operation missions in nuclear operations, bomb squad, disaster robots, and other domains with novel tasks or highly occluded environments could benefit from reduced manpower and teamwork demand, along with improved visual assistance quality based on trustworthy risk-aware motion in cluttered environments.
The ability to map challenging subarctic environments opens new horizons for robotic deployments in industries such as forestry, surveillance, and open-pit mining. In this paper, we explore the possibilities of large-scale lidar mapping in a boreal forest. Computational and sensory requirements with regards to contemporary hardware are considered as well. The lidar mapping is often based on the Simultaneous Localization and Mapping (SLAM) technique relying on pose graph optimization, which fuses the Iterative Closest Point (ICP) algorithm, Global Navigation Satellite System (GNSS) positioning, and Inertial Measurement Unit (IMU) measurements. To handle those sensors directly within the ICP minimization process, we propose an alternative approach of embedding external constraints. Furthermore, a novel formulation of a cost function is presented and cast into the problem of handling uncertainties from GNSS and lidar points. To test our approach, we acquired a large-scale dataset in the Forêt Montmorency research forest. We report on the technical problems faced during our winter deployments aiming at building 3D maps using our new cost function. Those maps demonstrate both global and local consistency over 4.1 km.
In this paper, we present a path planner for low-altitude terrain coverage in known environments with unmanned rotary-wing micro aerial vehicles (MAVs). Airborne systems can assist humanitarian demining by surveying suspected hazardous areas (SHAs) with cameras, ground-penetrating synthetic aperture radar (GPSAR), and metal detectors. Most available coverage planner implementations for MAVs do not consider obstacles and thus cannot be deployed in obstructed environments. We describe an open source framework to perform coverage planning in polygon flight corridors with obstacles. Our planner extends boustrophedon coverage planning by optimizing over different sweep combinations to find the optimal sweep path, and considers obstacles during transition flights between cells. We evaluate the path planner on 320 synthetic maps and show that it is able to solve realistic planning instances fast enough to run in the field. The planner achieves 14% lower path costs than a conventional coverage planner. We validate the planner on a real platform where we show low-altitude coverage over a sloped terrain with trees.
This paper highlights the need for a new power model for skid-steered wheeled robots driving on loose soil, and lays the groundwork to develop such a model. State-of-the-art power modeling assumes hard ground; under typical assumptions, this predicts constant power consumption over a range of small turning radii where the inner wheels rotate backwards. However, experimental results performed both in the field and in a controlled laboratory show that, on sand, power is not in fact constant with respect to turning radius in this case. Power peaks in a newly identified range of turns where the inner wheels rotate backwards but are dragged forward. Data shows higher motor torque and wheel sinkage in this range. A skidding wheel that is sunk into loose soil bulldozes a pile of sand; initial modeling of this phenomenon reproduces the trend in additional power with respect to turning radius. As work on a full power model for loose soil continues, this work identifies turning radii to avoid whenever possible in practice.
While most autonomous driving efforts reported are directed for general driving and mainly on major roads, there are numerous applications for autonomous vehicles for last mile mobility—from person mobility and mail delivery to flexible recharging of cars in parking structures. Over the last year, we have designed vehicles for the micro-mobility challenge. Our approach was based on adoption of the open-source Autoware system. The system was taken as a starting point for the design of a robust solution. Proposed requirements include a robust control design, a shift toward increased use of image data over LiDAR data, handling of a richer set of vehicles/pedestrians in a last mile scenario, and overall system characterization and evaluation. We present an overview of the overall design and the design decisions for construction of vehicles for last-mile delivery.
By installing omnidirectional measurement light detection and ranging (LiDAR) on a mobile robot, it is possible to acquire the surrounding environment. However, not all measurement areas of LiDAR can be used because of laser rays blocked by the robot’s body. Hence, in this research, the authors aim to expand the measurement area of LiDAR by installing mirrors at the blocked area to improve an ability of acquiring information regarding the front environment. The installation position and mirror angle can be determined from the desired area to be measured and the geometrical condition of the laser rays reflecting on the mirror. Moreover, based on the method, the robot enabled the collection of the surface information of stairs during stair climbing, which was typically difficult to gather.
This paper presents a system for the intelligent search of shipwrecks using Autonomous Underwater Vehicles (AUVs). It introduces a machine learning approach to the automatic identification of potential archaeological sites from AUV-obtained side-scan sonar (SSS) data. The site identification pipeline consists of a series of stages that set up for, run, and process the output of a convolutional neural network (CNN). To alleviate the issue of training data scarcity, i.e. the lack of SSS data that includes shipwrecks, and improve the performance at testing time, a data augmentation stage is included in the pipeline. In addition, edge detection and other traditional image processing feature extraction methods are used in parallel with CNN to improve algorithmic performance. Experiments from two multi-deployment shipwreck search expeditions involving actual AUV deployments along the coast of Malta for data collection and processing demonstrate the pipeline's usefulness. Results from these two field expeditions yielded a precision/recall of 29.34%/97.22% and 32.95%/80.39%, respectively. Despite the poor precision, the pipeline filters out 99.79% of the area in data set A and 99.31% of the area in data set B.
This paper presents the first-ever approach for autonomous 3D semantic mapping of coral reefs. The position of corals in 3D coordinates and the type of the coral are presented in such a3D semantic map. The intended application of this work is coral reef health monitoring, as the current assessment is based entirely on direct or indirect human observation. The proposed system joins a convolutional neural network (CNN) with a direct visual odometry approach and a correlation filter based tracker, Kernelized Correlation Filter (KCF), to identify the different coral species detected. In addition to the coral classification, the 3D position of each coral is identified producing a semantic map of the observed reef. Each coral is identified once and tracked to prevent a recount. The number of different coral species encountered in two separate traversed areas is reported. Furthermore, the shape and size of a coral can be extracted from the 3D reconstruction enabling the extraction of volumetric data for subsequent studies. Experimental results from the coral reefs of Barbados verify the robustness and accuracy of the proposed approach.
The ability to build structures with autonomous robots using only found, minimally processed stones would be immensely useful, especially in remote areas. Assembly planning for dry-stacked structures, however, is difficult since both the state and action spaces are continuous, and stability is strongly affected by complex friction and contact constraints. We propose a planning algorithm for such assemblies that uses a physics simulator to find a small set of feasible poses and then evaluates them using a hierarchical filter. We carefully designed the heuristics for the filters to match our goal of building stable, free-standing walls. These plans are then executed open-loop with a robotic arm equipped with a wrist RGB-D camera. Experimental results show that the proposed planning algorithm can significantly improve the state of the art in robotic dry stacking.
The past few decades have seen tremendous progress in planetary exploration rovers and the most groundbreaking missions in space exploration history. Although the main focus has been toward single rover deployment, there has also been substantial research into swarm intelligence and multi-rover systems. This paper presents single and multi-rover path planning strategies for future planetary exploration missions. The main objective is to develop real-time Coverage Path Planning (CPP) solutions in unknown environments. The algorithms presented here are based on simple building blocks that use solely myopic sensing information to iteratively compute a motion cost function and decide their next move. Several single rover exploration methods are introduced and compared through simulation. The results show that complete coverage is not always possible in planetary exploration scenarios. Also, by slightly reducing the required percent of terrain to explore, the efficiency can be improved. In addition to this, two variants of multi-rover path planning algorithms are defined. A first method is a fully collaborative method, where the rovers explore a region together and a second where the terrain is segmented and the rover operate separately from each other.
Long-term monitoring of natural environments raises significant challenges due to the strong perceptual aliasing in trees, bushes and shrubs. This paper reports on the multi-session localization and mapping of a small lake shore using an autonomous surface vessel equipped with a 2D lidar and a camera. Our publicly available dataset includes 130 autonomous surveys of the 1 km shoreline while recording lidar, GPS and image data. We build our globally consistent multi-session map using ICP at multiple scales. The end result is evaluated qualitatively by superimposing all the lidar maps, and quantitatively by comparing images taken from the same pose at different times. The localization and mapping results, as well as the dataset of image pairs, are made available within our public dataset.
Among the most challenging of environments in which an autonomous mobile robot might be required to serve is the subterranean environment. The complete lack of ambient light, unavailability of GPS, and geometric ambiguity make subterranean simultaneous localization and mapping (SLAM) exceptionally difficult. While there are many possible solutions to this problem, a visual-inertial framework has the potential to be fielded on a variety of robotic platforms which can operate in the spatially constrained and hazardous environments presented by the subterranean domain. In this work, we present an evaluation of visual-inertial SLAM in the subterranean environment with onboard lighting and show that it can consistently perform quite well, with less than 4% translational drift. However, this performance is dependent on including some modifications that depart from the typical formulation of VI-SLAM, as well as careful tuning of the system’s visual tracking parameters. We discuss the sometimes counter-intuitive effects of these parameters and provide insight into how they affect the system’s overall performance.
When a tracked robot moves in a volcanic environment, the robot often has to climb over unfixed obstacles such as rocks on the loose ground. On the other hand, tracked robots with sub-tracks have been proposed, and the climbing performance of these robots on fixed obstacles can be improved by optimally controlling the sub-tracks. However, the effect of sub-tracks on climbing over unfixed obstacles has not been clear. In this study, the effect of sub-tracks on the improvement of the climbing performance of tracked robots over unfixed obstacles was investigated. Specifically, the conditions that a tracked robot should meet in climbing over an unfixed cylindrical obstacle were derived. An experiment with a real robot revealed that the derived climbing-over conditions are valid, and that the climbing-over capability of the tracked robot can be increased by setting the sub-track angle optimally. Furthermore, the motion strategies of sub-tracks are discussed based on the experimental results.
This paper presents the development of a four-legged, four-wheel-drive hydraulic-powered rover that can traverse irregular terrains such as agricultural or forest landscapes while carrying manipulators, etc., to do heavy-duty work. The rover is equipped with driving wheels at the tip of its legs and passive wheels at its knees or the bottom of its belly. By controlling the joints' torque directly using hydraulic actuators, the contact forces at its rolling contact points can be optimally distributed according to the rover's states. This allows the acceleration of the rover's center of gravity and posture to be controlled while accommodating the terrain. This paper describes the mechanical design and control system of the rover, as well as the preliminary experiments. Although the torque control for the driving wheels is not yet successfully implemented, the experimental results using two rover prototypes imply the promising technological capacity of torque-controlled rovers with low-cost hydraulic components.
The observation of volcanoes is critical in the estimation of volcanic activities and disaster prevention. Volcanic gas from fumaroles is an effective target for these observations. This is because the gas is highly mobile and quickly transmits underground information to the surface. However, as typical volcanic gas has high temperature and contains toxic chemical components, the manned observation of volcanic fumaroles is dangerous. Therefore, in this research, the authors propose an improved version of the dual-wheeled robot to obtain volcanic information from around fumaroles. This robot has a passive tether-guide module to improve its turning range on a slope, which enables the robot to change its traversal direction independent of the tether direction; thus, it has improved turning motion, diagonal traversal motion, and crossing motion on a steep slope. Indoor and field experiments proved the validity of these mechanisms. The lessons learned from these field experiments are also discussed.
In Japan, the amount of agricultural production decreases year-by-year. Moreover, reduction in agricultural work population and increase of abandonment of cultivated land are major social issues. To overcome these issues, we have proposed a small agricultural robot “MY DONKEY®” which supports transportation of harvested crops and records the farm work of the user and crop yield in field map while moving closer to the user. In order to move in the furrowed field, it is necessary to detect ridges and furrows where the robot can move using robot-mounted sensors and follow the ridge while avoiding the ridges and crop rows. Furthermore, to realize smooth harvesting support, we propose a ridge-tracking control according to user’s behavior based on the recognition of the work contents of the user such as harvesting, loading of harvested crops to the robot, transportation, and movement. We propose the ridge-tracking control framework based on fuzzy set theory which can evaluate and integrate multiple situations and carry out experiments in strawberry farm.
To achieve high-accuracy tracking performance for wheeled mobile robots in spatially varying terrain conditions, it is necessary to estimate both the robot’s state and the slip conditions of the environment to a high degree of precision. The receding horizon estimation and control (RHEC) framework presents a systematic, adaptive optimisation approach to this problem, to which our prior work proposed a structured blocking (SB) extension to address performance limitations for motion both at high speeds and over varying terrain. In this work, we validate these results in a series of preliminary field experiments with the Swagbot platform, demonstrating performance improvements in position tracking of up to 7%, and up to 13% for speed tracking at speeds of 1.5 and 2.5 m/s.
Exploring the water-rich surface of Europa is a desirable planetary exploration objective for the coming decades. Knowledge of the Europan terrain is relatively uncertain and therefore to perform mobile surface operations a versatile mobility system capable of adapting its locomotion is desirable. Our work summarizes preliminary efforts to evaluate a variety of wheel-on-limb mobility modes on Europan analogue terrain populated with unstructured salt-evaporite formations. A redundant wheel-on-limb quadruped, RoboSimian, has been adapted to perform traverses over unstructured terrain with obstacles up to 50 cm tall (1.6 times its wheel diameter). Multiple mobility modes that are combinations of wheeled and limbed motion primitives are proposed and evaluated over varying roughness terrain as a function of energy usage. In combination with these mobility modes, a mode-agnostic actively articulated suspension control scheme that maintains ground traction, regulates chassis orientation and prevents suspension saturation is presented. The controller uses an inverse kinematics solver that leverages pre-computed lookup-tables for each limb. This is shown to be simple yet effective in the field and is able to scale to a robot with 32 degrees-of-freedom.
Ice is the most valuable resource on the Moon. It exists only at the poles where shadows are extensive and drivable routes are short. Robot routes to reach this ice are tenuous. Sun-synchronous lunar polar routes offer order-of-magnitude greater duration and range if such routes are achievable. Sun-synchrony is brittle in the sense that a rover must be at precisely scheduled time and place, so special localization techniques are warranted. Methods for terrain-based localization that work at equatorial regions are challenged at the lunar poles, where the grazing sunlight casts long shadows that obscure and change views over time. The shadows are shown here to accentuate craters as localization features. This paper presents a method that improves terrain registration at the poles of the Moon by probabilistically considering sensor and terrain uncertainty, and exploiting shadows as semantic features for localization. This method is validated and evaluated in simulated experiments.
In Japan, expectations for the automation of construction machines are increasing to solve the labor shortage in the construction industry. In this research, a robotization method by retrofitting a robot to conventional construction machines is introduced to lower the introduction barrier for regional construction companies. The target machine is a six-wheeled dump truck. With a retrofitted internal sensor unit and derived kinematics of six-wheeled articulated dump truck, a conventional Global Navigation Satellite System (GNSS)-based path tracking method was implemented on it. In addition, to ensure safety during operation, an emergency stop function was installed on the dump truck with three-dimensional Light Detection and Ranging (3D LiDAR). Initial experiments of forward and backward path tracking with an actual dump truck confirmed the validity of the method, and the maximum tracking error was 1 m. Further, in an emergency stop experiment, the dump truck detected the obstacle and stopped immediately after obstacle detection within the emergency-stop region, i.e., 25 m x 3 m in front of the dump truck. Based on the initial experiments, the authors concluded that even the retrofitted conventional dump truck could perform basic functions for autonomous driving, such as path tracking and emergency stop.
