Actuator faults in autonomous mobile robotic systems pose significant challenges, especially in unpredictable environments where system reliability is paramount. Fault tolerant control (FTC) strategies, particularly those leveraging actuator redundancy, have been explored to address these issues. However, traditional methods commonly rely on explicit fault diagnosis, which can be resource-intensive and challenging to implement accurately. This paper introduces a novel approach that combines deep reinforcement learning (DRL) with a linearised optimal model-based controller to achieve actuator fault recovery without explicit fault diagnosis. The integration of DRL within a model-based controller framework enhances system stability and fault recovery capabilities. The chosen application platform for this study is an autonomous underwater vehicle (AUV), where the partial or total failure of a mission critical component such as a thruster could jeopardise the success of the mission and potentially render the vehicle unrecoverable in the event of a fault. In this application case, a linear quadratic regulator (LQR) controller is employed as the model-based controller, while the soft actor-critic (SAC) algorithm is used as the DRL component to handle fault recovery. The DRL model is trained and evaluated in simulation before being directly applied to the physical AUV. The proposed method’s effectiveness is demonstrated through comparisons with a standard LQR controller, a conventional adaptive LQR controller and the proposed hybrid LQR-SAC controller. The results indicate that the LQR-SAC controller outperforms the standard and conventional adaptive LQR controllers in maintaining system performance under fault conditions, achieving a significant reduction in trajectory tracking error on a physical AUV.

Fish use their swim bladders to actively regulate their buoyancy, enabling them to move up and down through the water effortlessly with minimal energy expenditure - a capability that would be equally useful for soft underwater robots. Current buoyancy actuation mechanisms tend to either be large, rigid, use changes in temperature, are externally mounted, or involve the use of compressible fluids, limiting the environments and depth at which they can be used. In this paper, we set out to develop a novel buoyancy actuation method that regulates the ratio between higher and lower density incompressible fluids within an underwater soft robot, resulting in depth change. We define incompressible fluids as fluids that experience negligible compression when placed under high pressure in a fluidic environment. To validate the concept, a soft two-link robotic arm was developed and using our vision-based control system, the robot was able to reach target tip positions in our feasibility study. The buoyancy actuation method developed in this paper for a soft robotic arm has demonstrated for the first time that an entirely soft structure filled with incompressible fluids can change depth and has been mathematically shown to operate at great depths, such as 11000 metres.

This paper describes the problem of coordination of an autonomous Multi-Agent System which aims to solve the coverage planning problem in a complex environment. The considered applications are the detection and identification of objects of interest while covering an area. These tasks, which are highly relevant for space applications, are also of interest among various domains including the underwater context, which is the focus of this study. In this context, coverage planning is traditionally modelled as a Markov Decision Process where a coordinated MAS, a swarm of heterogeneous autonomous underwater vehicles, is required to survey an area and search for objects. This MDP is associated with several challenges: environment uncertainties, communication constraints, and an ensemble of hazards, including time-varying and unpredictable changes in the underwater environment. MARL algorithms can solve highly non-linear problems using deep neural networks and display great scalability against an increased number of agents. Nevertheless, most of the current results in the underwater domain are limited to simulation due to the high learning time of MARL algorithms. For this reason, a novel strategy is introduced to accelerate this convergence rate by incorporating biologically inspired heuristics to guide the policy during training. The PSO method, which is inspired by the behaviour of a group of animals, is selected as a heuristic. It allows the policy to explore the highest quality regions of the action and state spaces, from the beginning of the training, optimizing the exploration/exploitation trade-off. The resulting agent requires fewer interactions to reach optimal performance. The method is applied to the MSAC algorithm and evaluated for a 2D covering area mission in a continuous control environment.

Unmanned Aerial Vehicles (UAVs) play a crucial role in Wireless Sensor Network (WSN) applications, particularly in smart agriculture, enabling efficient data collection from large-scale sensor deployments. A key challenge in this field is optimizing the UAV’s flight path while considering real-world constraints such as UAV dynamics, sensor heterogeneity, communication ranges, hovering requirements, overlapping cluster heads, altitude variation, and UAV limitations. Many existing studies oversimplify this problem by considering some of these constraints while neglecting others, leading to suboptimal solutions. In this work, we propose a novel UAV path optimization algorithm, “OptiFly”, designed for WSN-based agricultural systems, incorporating all these constraints in a single framework. The optimization problem is formulated as a Nonlinear Programming (NLP) model and solved using an appropriate solver. Unlike existing approaches, OptiFly integrates UAV kinematics, dynamics, and aerodynamics into the optimization process while accounting for heterogeneous sensors, as commonly observed in real-world agricultural implementations. The algorithm ensures that the UAV hovers at an optimal position within each sensor’s coverage area, minimizing unnecessary movements. Additionally, we consider overlapping cluster heads, allowing the UAV to hover over their intersection regions and collect data from both at the same point, thereby minimizing travel distance. For sensors with small communication ranges, the UAV dynamically adjusts its altitude to maintain connectivity while conserving energy. Furthermore, OptiFly enables the UAV to be self-aware of its endurance, terminating the mission before exceeding its maximum flight time. Power consumption is also considered based on the UAV’s dynamics. Simulation results demonstrate that OptiFly significantly reduces both travel distance and energy consumption compared to unoptimized and optimized approaches. Additionally, the proposed algorithm proves to have low computational complexity in various scenarios. The achieved improvements validate the effectiveness of our method, making it a promising solution for UAV-WSN applications in smart agriculture and beyond.

This paper presents a methodology for the sim-to-real transfer of deep reinforcement learning (DRL) models for fault-tolerant control (FTC) in autonomous underwater vehicles (AUVs). The proposed approach leverages open-access simulation tools to train a DRL-based control reallocation strategy, enabling adaptive responses to actuator faults. Domain randomisation during DRL training is employed to enhance robustness against varying fault scenarios. The trained model is first validated in a second simulator to assess its adaptability across different simulation environments. It is then transferred to a physical AUV for real-world evaluation. Experimental results demonstrate that the DRL-based control strategy effectively maintains trajectory control despite actuator faults, outperforming standard controllers. This study highlights the feasibility of sim-to-real transfer of DRL-based FTC in underwater robotics.

Actuator faults in dynamic systems pose significant challenges, particularly for robotic systems operating in hostile environments such as Autonomous Underwater Vehicles (AUVs), risking loss of stability and performance degradation. Fault Tolerant Control (FTC) strategies, including Control Reallocation (CR), have been developed to mitigate such risks. However, these strategies extensively depend on explicit fault diagnosis, which may present challenges regarding computational demands and efficiency, particularly when dealing with unknown faults. This paper presents a novel method that performs CR with Deep Reinforcement Learning (DRL) for actuator fault recovery without explicit fault diagnosis. The approach is implemented on a BlueROV2 underwater vehicle and demonstrates improved performance for fault recovery compared to a standard Proportional-Integral-Derivative (PID) controller and a variable gain PID controller, both in simulation and in real-world conditions. The DRL-based CR method demonstrates generalisability by successfully handling faults not encountered during training, highlighting its adaptability to unforeseen circumstances.

Class incremental learning consists in training discriminative models to classify an increasing number of classes over time. However, doing so using only the newly added class data leads to the known problem of catastrophic forgetting of the previous classes. Recently, dynamic deep learning architectures have been shown to exhibit a better stability-plasticity trade-off by dynamically adding new feature extractors to the model in order to learn new classes followed by a compression step to scale the model back to its original size, thus avoiding a growing number of parameters. In this context, we propose a new algorithm that enhances the compression of previous class knowledge by cutting and mixing patches of previous class samples with the new images during compression using our Rehearsal-CutMix method. We show that this new data augmentation reduces catastrophic forgetting by specifically targeting past class information and improving its compression. Extensive experiments performed on the CIFAR and ImageNet datasets under diverse incremental learning evaluation protocols demonstrate that our approach consistently outperforms the state-of-the-art . The code will be made available upon publication of our work1.

This paper presents a novel actuation method to change the buoyancy of an underwater soft robot by pumping denser and less dense liquids, than the liquid the robot is immersed in, into the robot to actively change the mass of the robot and cause it to experience a change in buoyancy. The technological research gap lies in the method of pumping lighter and heavier fluids into a soft robot to cause it to experience a change in mass and depth, which has not been explored before to the best of the author's knowledge. An analysis of the forces that are placed on the robotic system and the necessary equations to determine the force produced by a solution with a particular ratio of solute to solvent are presented. Preliminary experiments were conducted to test the buoyancy-based actuation method discussed in this paper by building a two link, soft, inflatable robot arm. This robot was shown to change the floatation height of its links when denser fluid was pumped into its links.

Thruster faults are one of the most common malfunctions encountered during Unmanned Underwater Vehicle (UUV) missions. This type of fault can lead to unwanted behaviour and jeopardise the UUV mission. Successful thruster fault management depends on accurate diagnostics. However, some scenarios, particularly instances of thruster faults due to external factors, pose a hard diagnostic task. This is particularly challenging in the context of abnormal behaviours that are detected but no fault diagnosis can be provided by the onboard fault management system. This type of fault is called non-diagnosable and it is the main target of this work. The aim of this paper is to propose a solution for controlling UUVs subject to non-diagnosable thruster faults using a Deep Reinforcement Learning (DRL)-based approach. This paper provides a comparison between an end-to-end DRL-trained controller and a standard PID controller to overcome partial and total thruster faults of a UUV. The consistency and robustness of the proposed method is verified by simulations. The results demonstrate the DRL-based controller’s effectiveness in addressing non-diagnosable thruster faults that would otherwise hinder the successful completion of the mission.

Collisions at sea can have severe effects for humans and the environment, leading to costs for marine organisations. Human factors, particularly fatigue, contribute to more than 80% of maritime collisions. Ships and vessels have become increasingly sophisticated in design and more recently, autonomous navigation systems have been integrated to reduce human errors associated with collisions. The rise in maritime traffic makes these autonomous systems crucial for enhancing safety. A challenge for autonomous navigation systems, causing contention among mariners and within maritime literature, is compliance with COLREG (Convention on the International Regulations for Preventing Collisions at Sea). An emerging solution to this challenge involves the development of a light simulator (COLSim) that leverages historical AIS (Automatic Identification System) data to introduce multiple autonomous vehicles into real-world scenarios, while incorporating the COLREG rules to avoid collision. Acceptance of autonomous navigation systems is important for the future of transportation at sea and is the natural evolution of maritime traffic. As AI is still evolving, humans will continue to play a vital role in the operation of ships and vessels and interpretation of COLREGs. In the future, human-AI interactions will be important for work, health, and organisational safety, where both humans and AI can support each other. Finding a balance between human expertise and AI assistance is essential for the maritime industry's future safety and efficiency. Copyright (c) 2024 The Authors. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Unmanned Aerial Vehicle (UAV) path planning optimizes trajectory for efficient target reach with minimal energy consumption. This study enhances the Target Interception method (TI), to address wind-related challenges. We introduce two adaptations: TI Dynamic Path and TI External Influences. Implementing these methods in a dynamic wind environment yields results demonstrating enhanced reliability and efficiency. This research marks a significant stride in developing more effective UAV path planning algorithms.

Despite technological growth and worldwide advancements in various fields, the agriculture sector continues to face numerous challenges such as desertification, environmental pollution, resource scarcity, and the excessive use of pesticides and inorganic fertilizers. These unsustainable problems in agricultural field can lead to land degradation, threaten food security, affect the economy, and put human health at risk. To mitigate these global issues, it is essential for researchers and agricultural professionals to promote advancements in smart agriculture by integrating modern technologies such as Internet of Things (IoT), Unmanned Aerial Vehicles (UAVs), Wireless Sensor Networks (WSNs), and more. Among these technologies, this paper focuses on UAVs, particularly quadcopters, which can assist in each phase of the agricultural cycle and improve productivity, quality, and sustainability. With their diverse capabilities, quadcopters have become the most widely used UAVs in smart agriculture and are frequently utilized by researchers in various projects. To explore the different aspects of quadcopters’ use in smart agriculture, this paper focuses on the following: (a) the unique advantages of quadcopters over other UAVs, including an examination of the quadcopter types particularly used in smart agriculture; (b) various agricultural missions where quadcopters are deployed, with examples highlighting their indispensable role; (c) the modelling of quadcopters, from configurations to the derivation of mathematical equations, to create a well-modelled system that closely represents real-world conditions; and (d) the challenges that must be addressed, along with suggestions for future research to ensure sustainable development. Although the use of UAVs in smart agriculture has been discussed in other papers, to the best of our knowledge, none have specifically examined the most popular among them, “quadcopters”, and their particular use in smart agriculture in terms of types, applications, and modelling techniques. Therefore, this paper provides a comprehensive survey of quadcopters’ use in smart agriculture and offers researchers and engineers valuable insights into this evolving field, presenting a roadmap for future enhancements and developments.

Despite a significant proportion of the Earth being covered in water, exploration of what lies below has been limited due to the challenges and difficulties inherent in the process. Current state of the art robots such as Remotely Operated Vehicles (ROVs) and Autonomous Underwater Vehicles (AUVs) are bulky, rigid and unable to conform to their environment. This makes certain underwater regions, especially those characterised by tight and narrow crevasses - as is the case for coral reefs - inaccessible to currently available exploratory tools. Soft robotics offers potential solutions to this issue. Fluid-actuated eversion or growing robots, in particular, are a good example. While current eversion robots have found many applications on land, their inherent properties make them particularly well suited to underwater environments. An important factor when considering underwater eversion robots is the establishment of a suitable steering mechanism that can enable the robot to change direction as required. This project proposes a design for an eversion robot that is capable of steering while underwater, through the use of bending pouches, a design commonly seen in the literature on land-based eversion robots. These bending pouches contract to enable directional change. Similar to their land-based counterparts, the underwater eversion robot uses the same fluid in the medium it operates in to achieve extension and bending but also to additionally aid in neutral buoyancy. The actuation method of bending pouches meant that robots needed to fully extend before steering was possible. Three robots, with the same design and dimensions were constructed from polyethylene tubes and tested. Our research shows that although the soft eversion robot design in this paper was not capable of consistently generating the same amounts of bending for the inflation volume, it still achieved suitable bending at a range of inflation volumes and was observed to bend to a maximum angle of 68 degrees at 2000 ml, which is in line with the bending angles reported for land-based eversion robots in the literature. Copyright (C) 2024 The Authors. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Learning-based adaptive control methods hold the potential to empower autonomous agents in mitigating the impact of process variations with minimal human intervention. However, their application to autonomous underwater vehicles (AUVs) has been constrained by two main challenges: (1) the presence of unknown dynamics in the form of sea current disturbances, which cannot be modelled or measured due to limited sensor capability, particularly on smaller low-cost AUVs, and (2) the nonlinearity of AUV tasks, where the controller response at certain operating points must be excessively conservative to meet specifications at other points. Deep Reinforcement Learning (DRL) offers a solution to these challenges by training versatile neural network policies. Nevertheless, the application of DRL algorithms to AUVs has been predominantly limited to simulated environments due to their inherent high sample complexity and the distribution shift problem. This paper introduces a novel approach by combining the Maximum Entropy Deep Reinforcement Learning framework with a classic model-based control architecture to formulate an adaptive controller. In this framework, we propose a Sim-to-Real transfer strategy, incorporating a bio-inspired experience replay mechanism, an enhanced domain randomisation technique, and an evaluation protocol executed on a physical platform. Our experimental assessments demonstrate the effectiveness of this method in learning proficient policies from suboptimal simulated models of the AUV. When transferred to a real-world vehicle, the approach exhibits a control performance three times higher compared to its model-based nonadaptive but optimal counterpart.

This paper introduces a learning-based solution tailored for the integrated motion planning and control of Multiple Autonomous Underwater Vehicles (AUVs). Tackling the complexities of cooperative motion planning, encompassing tasks such as waypoint tracking and self/obstacle collision avoidance, becomes challenging in a rule-based algorithmic paradigm due to the diverse and unpredictable situations encountered, necessitating a proliferation of if-then conditions in the implementation. Recognizing the limitations of traditional approaches that are heavily dependent on models and geometry of the system, our solution ofers an innovative paradigm shift. This study proposes an integrated motion planning and control strategy that leverages sensor and navigation outputs to generate longitudinal and lateral control outputs dynamically. At the heart of this cutting-edge methodology lies a continuous action Deep Reinforcement Learning (DRL) framework, specifically based on the Twin Delayed Deep Deterministic Policy Gradient (TD3). This algorithm surpasses traditional limitations by embodying an elaborated reward function, enabling the seamless execution of control actions essential for maneuvering multiple AUVs. Through simulation tests under both nominal and perturbed conditions, considering obstacles and underwater current disturbances, the obtained results demonstrate the feasibility and robustness of the proposed technique. Copyright (c) 2024 The Authors. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Unmanned aerial vehicles (UAVs) play pivotal roles in various applications, from surveillance to delivery services. Efficient path planning for UAVs in dynamic environments with obstacles and moving landing stations is essential to ensure safe and reliable operations. In this study, we propose a novel approach that combines the A* algorithm with the grey wolf optimizer (GWO) for path planning, referred to as GW-A*. Our approach enhances the traditional A algorithm by incorporating weighted nodes, where the weights are determined based on the distance from obstacles and further optimized using GWO. A simulation using dynamic factors such as wind direction and wind speed, which affect the quadrotor UAV in the presence of obstacles, was used to test the new approach, and we compared it with the A* algorithm using various heuristics. The results showed that GW-A* outperformed A* in most scenarios with high and low wind speeds, offering more efficient paths and greater adaptability.

Deep Reinforcement Learning (DRL) techniques have received significant attention in control and decision-making algorithms. Most applications involve complex decision-making systems, justified by the algorithms' computational power and cost. While model-based versions are emerging, model-free DRL approaches are intriguing for their independence from models, yet they remain relatively less explored in terms of performance, particularly in applied control. This study conducts a thorough performance analysis comparing the data-driven DRL paradigm with a classical state feedback controller, both designed based on the same cost (reward) function of the linear quadratic regulator (LQR) problem. Twelve additional performance criteria are introduced to assess the controllers' performance, independent of the LQR problem for which they are designed. Two Deep Deterministic Policy Gradient (DDPG)-based controllers are developed, leveraging DDPG's widespread reputation. These controllers are aimed at addressing a challenging setpoint tracking problem in a Non-Minimum Phase (NMP) system. The performance and robustness of the controllers are assessed in the presence of operational challenges, including disturbance, noise, initial conditions, and model uncertainties. The findings suggest that the DDPG controller demonstrates promising behavior under rigorous test conditions. Nevertheless, further improvements are necessary for the DDPG controller to outperform classical methods in all criteria. While DRL algorithms may excel in complex environments owing to the flexibility in the reward function definition, this paper offers practical insights and a comparison framework specifically designed to evaluate these algorithms within the context of control engineering.

Autonomous vessels have emerged as a prominent and accepted solution. However, achieving full autonomy for marine vessels requires the development of robust and reliable mission planning and control systems that can handle various encounters with manned and unmanned vessels while operating effectively in various weather and sea conditions. These algorithms need to account for various aspects of a mission like global/local planning, manoeuvrability limitations, external disturbances, Collision Avoidance (COLAV), motion regulations. A significant challenge in this pursuit is ensuring the autonomous vessels' compliance with the International Regulations for Preventing Collisions at Sea (COLREGs). This paper proposes a (prototypical) realisation of an open-source simulator including replay of AIS Data. The aim of this simulator is to provide fast simulation feeded with real data and including autonomous agents. Copyright (c) 2024 The Authors. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/)

The LOTUS project aims at improving maritime surveillance. In this context, this position paper presents ongoing contributions, including novel machine learning algorithms for multi-agent systems to be applied to groups of underwater drones involved in surveillance missions. It emphasises incorporating human-machine teaming to bolster decision-making in maritime scenarios. The expected outcomes of this project comprise the robust control of groups of autonomous vehicles, adaptable to environmental changes, as well as an effective reporting method. Mission summaries will be delivered to human operators by way of narratives about the relevant events detected thanks to drones. The integration of this narrative construction powered by machine learning will enhance the overall effectiveness of the team, constituting a significant breakthrough.