Search and rescue personnel is facing many challenges when deployed in the field after a natural or man-made disaster. In some cases they are exposed to safety risks, for instance when searching for trapped victims under a partially collapsed building after an earthquake. Robots could be a tool that the search and rescue teams could use to search in areas that are too dangerous or too difficult to reach. In this paper, part of the effort made by the CURSOR project is described. In particular, we present a software architecture designed and developed for the Soft Miniaturised Underground Robotic Finder (SMURF). The SMURF is a robotic platform designed and built to assist the search and rescue teams during their operations. Finally, we describe the main components of the SMURFs and share our findings and our acquired experience when developing and testing the SMURFs in realistic environments.

This paper presents the main results and latest developments in a 4-year project called autonomous subsea intervention (SEAVENTION). In the project we have developed new methods for autonomous inspection, maintenance and repair (IMR) in subsea oil and gas operations with Unmanned Underwater Vehicles (UUVs). The results are also relevant for offshore wind, aquaculture and other industries. We discuss the trends and status for UUV-based IMR in the oil and gas industry and provide an overview of the state of the art in intervention with UUVs. We also present a 3-level taxonomy for UUV autonomy: mission-level, task-level and vehicle-level. To achieve robust 6D underwater pose estimation of objects for UUV intervention, we have developed marker-less approaches with input from 2D and 3D cameras, as well as marker-based approaches with associated uncertainty. We have carried out experiments with varying turbidity to evaluate full 6D pose estimates in challenging conditions. We have also devised a sensor autocalibration method for UUV localization. For intervention, we have developed methods for autonomous underwater grasping and a novel vision-based distance estimator. For high-level task planning, we have evaluated two frameworks for automated planning and acting (AI planning). We have implemented AI planning for subsea inspection scenarios which have been analyzed and formulated in collaboration with the industry partners. One of the frameworks, called T-REX demonstrates a reactive behavior to the dynamic and potentially uncertain nature of subsea operations. We have also presented an architecture for comparing and choosing between mission plans when new mission goals are introduced.

To get a better understanding of the highly nonlinear processes driving the ocean, efficient and informative sampling is critical. By combining robotic sampling with ocean models we are able to choose informative sampling sites and adaptively change our path based on measurements. We present models exploiting prior information from ocean models as well as real-time information from in situ measurements. The method uses compact Gaussian process modeling and objective functions to locate informative sampling sites. Our aim is to get a better understanding of ocean processes and improve real-time monitoring of dispersal dynamics. The case study focuses on a fjord located in Norway containing a seafill for mine tailings. Transportation of the deposited particles are studied, and the sampling method is tested in the area. The results from these sea trials are presented.

Our research concerns the coordination and control of robotic vehicles for upper water-column oceanographic observations. In such an environment, operating multiple vehicles to observe dynamic oceanographic phenomena, such as ocean processes and marine life, from fronts to cetaceans, has required that we design, implement and operate software, methods and processes which can support opportunistic needs in real-world settings with substantial constraints. In this work, an approach for coordinated measurements using such platforms, which relate directly to task outcomes, is presented. We show the use and operational value of a new Artificial Intelligence based mixed-initiative system for handling multiple platforms along with the networked infrastructure support needed to conduct such operations in the open sea. We articulate the need and use of a range of middleware architectures, critical for such deployments and ground this in the context of a field experiment in open waters of the mid-Atlantic in the summer of 2015.

Our research concerns the mixed-initiative coordination of air and underwater vehicles interacting over inter-operated radio and underwater communication networks for novel oceanographic field studies. In such an environment, operating multiple vehicles to observe dynamic oceanographic events such as fronts, plumes, blooms and cetaceans has required that we design, implement and operate software, methods and processes which can support ephemeral and unpredictable observations (including those of moving animals) in real-world settings with substantial constraints. We articulate an approach for coordinated measurements using such platforms, which relate directly to task outcomes. We show the use and operational value of a new Artificial Intelligence (AI) based mixed-initiative system, EUROPtus, for handling multiple platforms from a recent field experiment in open waters of the mid-Atlantic.

Ocean fronts are productivity hot spots, supporting marine life from plankton to whales. These dynamic systems contain a vast amount of information, and have the potential to significantly expand our knowledge of aquatic ecosystems in relation to climate change. However, ocean fronts and other dynamic features cannot be studied through conventional oceanographic techniques. In this paper, we address the problem of sampling and tracking an ocean front with an Autonomous Underwater Vehicle based on predictions and/or priors provided by a heterogeneous team of assets and ocean models. Specifically, given a prior (that may not be accurate or up-to-date) we present a method for an underwater vehicle to plan a mission and adapt this mission on-the-fly to track a dynamic feature. Results from field trials are presented, and demonstrate that the vehicle is able to adapt its path to follow a desired contour.

Over the last decade, ocean sunfish movements have been monitored worldwide using various satellite tracking methods. This study reports the near-real time monitoring of fine-scale (< 10 m) behaviour of sunfish. The study was conducted in southern Portugal in May 2014 and involved satellite tags and underwater and surface robotic vehicles to measure both the movements and the contextual environment of the fish. A total of four individuals were tracked using custom-made GPS satellite tags providing geolocation estimates of fine-scale resolution. These accurate positions further informed sunfish areas of restricted search (ARS), which were directly correlated to steep thermal frontal zones. Simultaneously, and for two different occasions, an Autonomous Underwater Vehicle (AUV) video-recorded the path of the tracked fish and detected buoyant particles in the water column. Importantly, the densities of these particles were also directly correlated to steep thermal gradients. Thus, both sunfish foraging behaviour (ARS) and possibly prey densities, were found to be influenced by analogous environmental conditions. In addition, the dynamic structure of the water transited by the tracked individuals was described by a Lagrangian modelling approach. The model informed the distribution of zooplankton in the region, both horizontally and in the water column, and the resultant simulated densities positively correlated with sunfish ARS behaviour estimator (rs = 0.184, p<0.001). The model also revealed that tracked fish opportunistically displace with respect to subsurface current flow. Thus, we show how physical forcing and current structure provide a rationale for a predator’s fine-scale behaviour observed over a two weeks in May 2014.

Supervision and control of Autonomous underwater vehicles (AUVs) has traditionally been focused on an operator determining a priori the sequence of waypoints of a single vehicle for a mission. As AUVs become more ubiquitous as a scientific tool, we envision the need for controlling multiple vehicles which would impose less cognitive burden on the operator with a more abstract form of human-in-the-loop control. Such mixed-initiative methods in goal-oriented commanding are new for the oceanographic domain and we describe the motivations and preliminary experiments with multiple vehicles operating simultaneously in the water, using a shore-based automated planner.

Robotic sampling is attractive in many field robotics applications that require persistent collection of physical samples for ex-situ analysis. Examples abound in the earth sciences in studies involving the collection of rock, soil, and water samples for laboratory analysis. In our test domain, marine ecosystem monitoring, detailed understanding of plankton ecology requires laboratory analysis of water samples, but predictions using physical and chemical properties measured in real-time by sensors aboard an autonomous underwater vehicle (AUV) can guide sample collection decisions. In this paper, we present a data-driven and opportunistic sampling strategy to minimize cumulative regret for batches of plankton samples acquired by an AUV over multiple surveys. Samples are labeled at the end of each survey, and used to update a probabilistic model that guides sampling during subsequent surveys. During a survey, the AUV makes irrevocable sample collection decisions online for a sequential stream of candidates, with no knowledge of the quality of future samples. In addition to extensive simulations using historical field data, we present results from a one-day field trial where beginning with a prior model learned from data collected and labeled in an earlier campaign, the AUV collected water samples with a high abundance of a pre-specified planktonic target. This is the first time such a field experiment has been carried out in its entirety in a data-driven fashion, in effect “closing the loop” on a significant and relevant ecosystem monitoring problem while allowing domain experts (marine ecologists) to specify the mission at a relatively high level.

This paper describes a command and control infrastructure that provides mixed- initiative user interfaces for the coordination and control of networked vehicle systems composed by heterogeneous robots, satellite trackers, manned ships and communication gateways. The presented infrastructure was tested in a field experiment for tracking and sampling the surrounding of Ocean Sunfish. In this experiment, the system was used by operators to obtain global situation awareness and to specify high-level objectives that were incorporated by a shore-side planner and adapted during execution onboard the vehicles.

Obtaining synoptic observations of dynamic ocean phenomena such as fronts, eddies, oxygen minimum zones and blooms has been challenging primarily due to the large spatial scales involved.Traditional methods of observation with manned ships are expensive and, unless the vessel can survey at high-speed, unrealistic.Autonomous underwater vehicles (AUVs) are robotic platforms that have been making steady gains in sampling capabilities and impacting oceanographic observations especially in coastal areas.However, their reach is still limited by operating constraints related to their energy sources.Unmanned aerial vehicles (UAVs) recently introduced in coastal and polar oceanographic experiments have added to the mix in observation strategy and methods.They offer a tantalizing opportunity to bridge such scales in operational oceanography by coordinating with AUVs in the water-column to get in-situ measurements.In this paper, we articulate the principal challenges in operating UAVs with AUVs making synoptic observations for such targeted watercolumn sampling.We do so in the context of autonomous control and operation for networked robotics and describe novel experiments while articulating the key challenges and lessons learned.

Marine phenomena such as algal blooms can be detected using in situ measurements onboard autonomous underwater vehicles (AUVs), but understanding plankton ecology and community structure requires retrieval and analysis of water specimens. This process requires shipboard or manual sample collection, followed by onshore lab analysis which is time-consuming. Better understanding of the relationship between the observable environmental features and organism abundance would allow more precisely targeted sampling and thereby save time. In this work, we present an approach to learn and improve models that predict this relationship. Coupled with recent advances in AUV technology allowing selective retrieval of water samples, this constitutes a new paradigm in biological sampling. We use organism abundance models along with spatial models of environmental features learned immediately after AUV deployments to compute spatial distributions of organisms in the coastal ocean purely from in situ AUV data. We use Gaussian process regression along with the unscented transform to fuse the two models, obtaining both the mean and variance of the organism abundance estimates. The uncertainty in organism abundance predictions is used in a sampling strategy to selectively acquire new water specimens that improves the organism abundance models. Simulation results are presented demonstrating the advantage of performing hierarchical probabilistic regression. After the validation through simulation, we show predictions of organism abundance from models learned on lab-analyzed water sample data, and AUV survey data.

This paper describes scientific and engineering efforts towards remote sensing, tagging and continuous tracking of sunfish specimens (Mola mola, world largest bony fish) in the open sea.This ongoing experiment is done in the context of a collaboration between MBARI and Porto University and bridges biology, fisheries research, autonomy and control with use of satellite tags, autonomous aerial, surface and underwater vehicles, as well as new software tools for remote asset control and situational awareness.Here we describe technologies and overall architecture of the experiment and results from initial deployments.

We consider the problem of tracing the structure of oceanological features using autonomous underwater vehicles (AUVs). Solving this problem requires the construction of a control strategy that will determine the actions for the AUV based on the current state, as measured by on-board sensors and the historic trajectory (including sensed data) of the AUV. We approach this task by applying plan-based policy-learning, in which a large set of sampled problems are solved using planning and then, from the resulting plans a decision-tree is learned, using an established machine-learning algorithm, which forms the resulting policy. We evaluate our approach in simulation and report on sea trials of a prototype of a learned policy. We indicate some of the lessons learned from this deployed system and further evaluate an extended policy in simulation.

In the coastal ocean, biological and physical dynamics vary on spatiotemporal scales spanning many orders of magnitude. At large spatial (O(100km)) and temporal (O(weeks to months)) scales, traditional shipboard and moored measurements are very effective at quantifying mean and varying oceanic properties. At scales smaller than the internal Rossby radius (O(10km) for typical coastal stratification at mid-latitude), horizontal, vertical and temporal inhomogeneity is the rule rather than the exception.

The coastal ocean is a dynamic and complex environment due to the confluence of atmospheric, oceanographic, estuarine/riverine, and land-sea interactions. Yet it continues to be undersampled, resulting in poor understanding of dynamic, episodic, and complex phenomena such as harmful algal blooms, anoxic zones, coastal plumes, thin layers, and frontal zones. Often these phenomena have no viable biological or computational models that can provide guidance for sampling. Returning targeted water samples for analysis becomes critical for biologists to assimilate data for model synthesis. In our work, the scientific emphasis on building a species distribution model necessitates spatially distributed sample collection from within hotspots in a large volume of a dynamic field of interest. To do so, we propose an autonomous approach to sample acquisition based on an online calculation of sample utility. A series of reward functions provide a balance between temporal and spatial scales of oceanographic sampling and do so in such a way that science preferences or evolving knowledge about the feature of interest can be incorporated in the decision process. This utility calculation is undertaken onboard a powered autonomous underwater vehicle (AUV) with specialized water samplers for the upper water column. For validation, we provide experimental results using archival AUV data along with an at-sea demonstration in Monterey Bay, CA.

Fronts have been recognized as hotspots of intense biological activity and are important targets for observation to understand coastal ecology and transport in a changing ocean. With high spatial and temporal variability, detection and event response for frontal zones is challenging for robotic platforms like autonomous underwater vehicles (AUVs). These vehicles have shown their versatility and cost-effectiveness in using automated approaches to detect a range of features. Targeting them for in-situ observation and sampling capabilities for frontal zones then provides an important tool for characterizing rapid and episodic changes. We introduce a novel momentum-based front detection (MBFD) algorithm which utilizes a Kalman filter and a momentum accumulator function to identify significant temperature gradients associated with upwelling fronts. MBFD is designed to work at a number of levels including onboard an AUV, on-shore with a sparse real-time data stream and post-experiment on a full resolution data set gathered by a vehicle. Such a multi-layered approach plays an important role in mixed human-robot decision making for oceanographers making coordinated sampling and asset allocation strategies in large multi-robot field experiments in the coastal ocean.

We extend existing oceanographic sampling methodologies to sample an advecting feature of interest using autonomous robotic platforms. GPS-tracked Lagrangian drifters are used to tag and track a water patch of interest with position updates provided periodically to an autonomous underwater vehicle (AUV) for surveys around the drifter as it moves with ocean currents. Autonomous sampling methods currently rely on geographic waypoint track-line surveys that are suitable for static or slowly changing features. When studying dynamic, rapidly evolving oceanographic features, such methods at best introduce error through insufficient spatial and temporal resolution, and at worst, completely miss the spatial and temporal domain of interest. We demonstrate two approaches for tracking and sampling of advecting oceanographic features. The first relies on extending static-plan AUV surveys (the current state-of-the-art) to sample advecting features. The second approach involves planning of surveys in the drifter or patch frame of reference. We derive a quantitative envelope on patch speeds that can be tracked autonomously by AUVs and drifters and show results from a multi-day off-shore field trial. The results from the trial demonstrate the applicability of our approach to long-term tracking and sampling of advecting features. Additionally, we analyze the data from the trial to identify the sources of error that affect the quality of the surveys carried out. Our work presents the first set of experiments to autonomously observe advecting oceanographic features in the open ocean.