Reinforcement learning (RL) techniques have increasingly been integrated into fault diagnosis (FD) and fault-tolerant control (FTC) systems due to their robust feature representation capabilities and adaptability. To facilitate related research, this paper presents a synthesis of recent progress in reinforcement learning methodologies for FD and FTC systems. Initially, it outlines key concepts and formulations of RL systems. Subsequently, it reviews the commonly employed RL architectures, with particular emphasis on model-free, model-based and deep learning methods. Finally, it delves into the challenges faced in current applications of RL-based FD and FTC, covering aspects like imbalanced data, safety and robustness, interpretability and explainability, along with potential solutions. This paper strives to offer a comprehensive roadmap for furthering RL-based FD and FTC research within the community.

Accurate fault detection and diagnosis are critical components of any fault-tolerant control system, especially for unmanned aerial vehicles (UAVs) where reliability is paramount. Traditionally, both model-based and data-driven approaches have been applied for fault diagnosis. However, the increasing complexity of high-dimensional UAV systems has shifted focus toward data-driven methods, which leverage advanced classification algorithms to enhance fault identification and isolation. This study builds on this evolution by developing a sophisticated condition-based monitoring (CBM) system specifically designed for multirotor UAVs. In contrast to earlier studies that primarily relied on raw data for classifier training, this work introduces advanced preprocessing techniques and multidomain feature extraction, significantly improving the robustness and accuracy of fault detection. A comparative analysis is performed between feature-selection methods, including recursive feature elimination with cross-validation (RFECV) and variational autoencoder (VAE), to extract critical insights into UAV operational behavior. Through testing and evaluating various classification models on data from a hexarotor UAV under diverse actuator fault conditions, this research identifies optimal approaches for real-time fault detection and diagnosis. Results demonstrate notable improvements across all evaluation metrics, establishing this approach as a substantial advancement in UAV fault tolerance.

This paper presents a novel Active Fault-Tolerant Control (AFTC) framework for a four-wheel drive (4WD) electric vehicle equipped with independently actuated in-wheel motors (IWMs). The presented approach consists of a fault detection and diagnosis (FDD) module and a compensation strategy. Once a fault is detected, the FDD module is activated, and as a consequence the fault will be identified, the faulty wheel will be isolated, and fault magnitude will be estimated. Then, based on the FDD module outputs, compensation module strategy is initiated. Compensation module employs a multi-parametric optimization technique to achieve the main objective of reducing the torque demand to the faulty actuator. Through extensive MATLAB/Simulink simulations, the results of this study showcase the effectiveness of the proposed AFTC system in managing multiplicative faults affecting the IWMs of the electric vehicle.

Load forecasting is essential for efficient microgrid management, providing key advantages in operational efficiency, cost control, and grid reliability. As microgrids become increasingly critical in the global transition toward decentralized renewable energy systems, accurately predicting load demand is vital for optimizing performance and ensuring a stable, resilient, and sustainable power supply. This study introduces a novel short-term load forecasting approach based on Belief Functions Theory (BFT). The proposed method employs information fusion techniques to combine multiple predictors, each with its own forecasting mechanism. Using lagged power values and weather data, the predictors generate estimated power values along with corresponding uncertainty or error levels. A mass function is assigned to each predictor, taking into account both prediction and error data, even when some information is missing. These mass functions are then merged to produce a final, reliable prediction. Application of this method to publicly available load datasets demonstrates its effectiveness, achieving a 12% reduction in forecasting error compared to state-of-the-art methods and delivering substantial improvements in computational efficiency.

Actuator faults in unmanned aerial vehicles (UAVs) can have significant and potentially adverse effects on their safety and performance, highlighting the critical importance of fault diagnosis in UAV design. Ensuring the reliability of these systems in various applications often requires the use of advanced diagnostic algorithms. Artificial intelligence methods, such as deep learning and machine learning techniques, enable fault diagnosis through sample-based learning without the need for prior knowledge of fault mechanisms or physics-based models. However, UAV fault datasets are typically small due to stringent safety standards, which presents challenges for achieving high-performance fault diagnosis. To address this, deep reinforcement learning (DRL) algorithms offer a unique advantage by combining deep learning’s automatic feature extraction with reinforcement learning’s interactive learning approach, improving both learning capabilities and robustness. In this study, we propose and evaluate two DRL-based fault diagnosis models, which demonstrate remarkable accuracy in fault diagnosis, consistently exceeding $99{\rm{\% }}$ . Notably, under small sample scenarios, the proposed models significantly outperform traditional classifiers such as decision trees, support vector machines, and multilayer perceptron neural networks. These findings suggest that the integration of DRL enhances fault diagnosis performance, particularly in data-limited environments.

A major limitation of current battery-powered Unmanned Aerial Vehicles (UAVs) is their finite onboard energy, which restricts both flight duration and the range of missions they can effectively complete. To overcome this challenge and extend flight times, reducing the vehicle's energy consumption is crucial. Various strategies have been explored in the literature to achieve this, including advancements in UAV design and materials, the incorporation of energy-capturing systems, and the development of energy-minimizing control approaches. This work focuses on optimizing control strategies to minimize energy consumption. Specifically, the aim is to improve the efficiency of a hexarotor supplied by a Lithium-Polymer (Li-Po) battery as it follows a designated reference position by introducing innovative high-level energy-saving control techniques. To achieve this, three metaheuristic algorithms are developed to address the optimization problem: Grey Wolf Optimizer, Particle Swarm Optimization and Genetic Algorithm. All methods are implemented and validated using the MATLAB/Simulink simulation environment.

In recent times, there has been a notable increase in the use of Unmanned Aerial Vehicles (UAVs) worldwide, driven by a growing demand for their versatile applications across various domains. However, alongside their beneficial applications, there has been a concerning emergence of malicious UAVs use by cyber criminals. These unauthorized activities pose significant risks with the potential for destructive consequences. Consequently, there is a pressing need for the development and implementation of detection, protection, and prevention measures to mitigate these threats effectively. The primary objective of this paper is to explore the evolving risks associated with cyber-attacks in formation flights of UAVs along with the corresponding countermeasures aimed at tolerating such threats. The work proposes a hybrid fault diagnosis scheme and fault-tolerant cooperative controllers for multiple UAVs under faults and cyber attacks. The proposed hybrid fault diagnosis scheme combines rule-based and model-based approaches. Three realistic attack scenarios are simulated including the man-in-the-middle attack and the GPS spoofing. The results show that the proposed scheme is able to ensure the safe operation of UAVs in the fleet by effectively diagnosing faults and enabling proactive measures to mitigate potential risks.

In recent years, advanced deep learning techniques have emerged as pivotal tools in enabling the development of robust vision-based solutions for steel surface inspection. This resulted in enhanced inspection accuracy, all while significantly reducing costs in the manufacturing industry. However, the lack of actual steel surface defects datasets currently places a certain constraint on further research into classifying those anomalies. As a consequence, the Convolutional Neural Network (CNN) technique, known for its prowess in image-related tasks, faces certain challenges, especially in classifying less common defects. This work proposes a novel hybrid CNN model with a Support Vector Machine (SVM) classifier at the output layer for surface defects classification. The features extracted from the pre-trained ResNet152 and EfficientB0 CNN algorithms are concatenated and fed to the SVM layer for classification. Extensive experiments on a merged dataset consisting of the publicly available Northeastern University (NEU) dataset and Xsteel surface defect dataset (X-SDD) are carried out and the accuracy and F1 scores are calculated for performance evaluation. The merged dataset contains eleven typical defect types with a total of 2660 defect images. Then, the adopted algorithm is compared with ten fine-tuned deep learning models to evaluate the performance of transfer learning for steel defect detection and identification. The evaluation results show that the deep feature extraction and SVM classification produced better results than the transfer learning. Finally, the proposed classifier model is validated on a newly collected dataset from a Computed Tomography scanner with an accuracy reaching over 96%.

This work introduces a novel fault-tolerance technique for data fusion in Unmanned Aerial Vehicles (UAVs), designed to address sensor faults through a deep learning-based framework. Unlike traditional methods that rely on hardware redundancy, our approach leverages Long Short-Term Memory (LSTM) networks for state estimation and a moving average (MA) algorithm for fault detection. The novelty of our technique lies in its dual strategy: utilizing LSTMs to analyze residuals and detect errors, while the MA algorithm identifies faulty sensors by monitoring variations in sensor data. This method allows for effective error correction and system recovery by replacing faulty measurements with reliable ones, eliminating the need for a fault-free prediction model. The approach has been validated through offline testing on real sensor data from a hexarotor UAV with simulated faults, demonstrating its efficacy in maintaining robust UAV operations without resorting to redundant hardware solutions.

Recent advancements in autonomous driving technologies have significantly enhanced road safety and collision avoidance. However, ensuring this in dynamic driving environments remains a challenging endeavor. This paper addresses this challenge by proposing a high-level risk-aware decision-making module integrated into the trajectory planner of autonomous vehicles. The module establishes a function for dynamic risk assessment, considering both longitudinal and lateral aspects of the environment. By incorporating various factors such as velocity and relative position, the proposed function enables the vehicle to anticipate and respond to potential hazards proactively. Additionally, the paper incorporates and builds upon previous work of a modular and distinctive AI-based Adaptive Cruise Control (ACC) system with robust generalization capabilities. Results demonstrate the effectiveness of the proposed approach in improving safety, collision avoidance, and adaptability in highly interactive driving environments allowing the vehicle to dynamically re-plan trajectories and speed profiles during lane change maneuvers.

The past decades have witnessed an increase in the use of multirotor unmanned aerial vehicles (UAVs). This has led the way to the development of fault-tolerant control algorithms to service the different applications provided by these UAVs in different situations and circumstances. This paper is concerned with theoretical approaches and real practices for Fault Detection and Diagnosis (FDD) and Fault-Tolerant Control (FTC) of multirotor UAVs, namely quadrotors, hexarotors and octorotors. The topic is addressed by identifying the most significant faults and failures on these vehicles, the different FDD and FTC architectures as well as the experimental real-time validations available in the published literature. A review of some challenges in the general area of multirotor UAV fault-tolerant control design is further concluded.

In the context of global warming, renewable energy sources, particularly wind and solar power, have garnered increasing attention in recent decades. Accurate forecasting of the energy output in microgrids (MGs) is essential for optimizing energy management, reducing maintenance costs, and prolonging the lifespan of energy storage systems. This study proposes an innovative approach to solar irradiance forecasting based on the theory of belief functions, introducing a novel and flexible evidential method for short-to-medium-term predictions. The proposed machine learning model is designed to effectively handle missing data and make optimal use of available information. By integrating multiple predictive models, each focusing on different meteorological factors, the approach enhances forecasting accuracy. The Yager combination method and pignistic transformation are utilized to aggregate the individual models. Applied to a publicly available dataset, the method achieved promising results, with an average root mean square error (RMS) of 27.83 W/m2 calculated from eight distinct forecast days. This performance surpasses the best reported results of 30.21 W/m2 from recent comparable studies for one-day-ahead solar irradiance forecasting. Comparisons with deep learning-based methods, such as long short-term memory (LSTM) networks and recurrent neural networks (RNNs), demonstrate that the proposed approach is competitive with state-of-the-art techniques, delivering reliable predictions with significantly less training data. The full potential and limitations of the proposed approach are also discussed.

In this paper, we propose a novel adaptive feedforward super-twisting sliding mode control algorithm to resolve the tracking control problem of parallel manipulators. The proposed control scheme includes three main terms, (i) the standard super-twisting algorithm, (ii) an adaptive feedforward dynamic model, and (iii) a feedback term to ensure stability. The proposed controller provides robustness towards uncertainties and disturbances, less sensitive to measurement noise, and allows dynamic parameters adaptation of the manipulator while executing a certain task. Real-time experiments are conducted on a 3-DOF non-redundant Delta parallel robot, including two main scenarios, (i) nominal case, and (ii) robustness towards operating acceleration changes. The relevance of the proposed controller is verified experimentally in both scenarios and compared with two other controllers from the literature, including the standard and the feedforward super-twisting sliding mode control algorithms.

Although important improvements in the area of robust control of nonlinear systems have been presented in the literature, most of the developed controllers suffer from complexity and large dependency on accurate mathematical formulation of the models. Recently, model-free robust control techniques were introduced and have shown good performance when applied to multi-input–multi-output systems. The model-free approach is characterized by the nonuse of any prior knowledge about the underlying structure and/or associated parameters of the dynamical system. Therefore, the major criteria for assessing the effectiveness of these controllers are related to their ability to handle unknown inputs and disturbances, as well as achieving the desired tracking performance in presence of faults and malfunctions. This work considers the development of robust fault-tolerant controllers based on the model-free approach and their application to multirotor unmanned aerial vehicles’ (UAVs) systems. The different controllers based on intelligent proportional-derivative (iPD), intelligent backstepping (iBackstepping), and adaptive control are compared in terms of performance, ease of implementation and parameters tuning. The simulated results, tested on Matlab/Simulink on a full nonlinear model of a hexarotor UAV, validate the theoretical advantages of the adaptive approach with respect to multiple criteria such as improved tracking performance in case of existence of actuators faults when compared to the iPD and iBackstepping control methods at the cost of increased complexity.

Flight endurance of an electric-powered Unmanned Aerial Vehicle (UAV) is restricted by its limited on-board energy, which might endanger the accomplishment of any mission it has been sent on. Therefore, the vehicle’s energy consumption should be reduced as much as possible to prolong its flight. This paper deals with the minimization of the energy consumed by an electric-powered UAV as it tracks a desired target point by designing the necessary energy saving control techniques. Two approaches are proposed. The first one addresses the energy consumption problem by optimizing the UAV’s low-level controller. The second approach makes use of the powerful Artificial Bee Colony algorithm to optimize the medium-level controller which guarantees not only target point tracking but also minimization of energy consumption. The effectiveness of both proposals is validated by simulations on MATLAB/Simulink.

This article deals with the development and implementation of a novel feedforward super-twisting sliding mode controller for robotic manipulators. A full stability analysis based on a Lyapunov candidate is established showing a local asymptotic finite-time convergence of the proposed controller in the presence of upper bounded disturbances. Its robustness toward parametric uncertainties and system disturbances, thanks to the super-twisting approach, is pointed out. In addition, the feedforward dynamic term of the proposed controller that can compensate for the model nonlinearities is not sensitive to measurement noise. Real-time experiments have been conducted on two parallel manipulators: a 5-DOF SPIDER4 PKM and a 3-DOF Delta PKM. The effectiveness of the proposed controller is validated in different scenarios, including the nominal case and robustness toward parametric variations (payload) and speed changes

In the evolving automobile industry, Adaptive Cruise Control (ACC) is key for aiding autonomous traffic navigation. Ideal ACC systems can decelerate to low speeds in stop-and-go traffic, maintain a safe following distance, minimize rear-end collision risks, and lessen the driver's need to continually adjust vehicle's speed to match traffic flow. In this paper, we offer a Deep Reinforcement Learning-based adaptive cruise control (DRL-ACC) system that creates safe, flexible, and responsive car-following policies agents. Instead of using discrete incremental and decremental values or a continuous action space, we suggest constructing a discrete high-level action space to accelerate, decelerate, and hold the current speed. We also provide a comprehensive, easy-to-interpret multi-objective reward function that reflects safe, responsive, and rational traffic behavior. This strategy, trained on a single steady-state flow car-following scenario, promotes steadiness, responsiveness, and shows better generalization to diverse car-following scenarios. Results are also compared to the conventional Intelligent Driver Model (IDM). We further explore the model's potential to avoid rear-end collisions and facilitate future integration of lane-change maneuvers, which will increase its effectiveness in emergency situations.

Statistical machine learning classification methods have been widely used in the fault detection analysis in several engineering domains. This motivates us to provide in this article an overview on the application of these methods in the fault diagnosis strategies and also their successful use in unmanned aerial vehicles (UAVs) systems. Different existing aspects including the implementation conditions, offline design, and online computation algorithms as well as computation complexity and detection time are discussed in detail. Evaluation and validation of these aspects have been ensured by a simple demonstration of the basic classification methods and neural network techniques in solving the fault detection and diagnosis problem of the propulsion system failure of a multirotor UAV. A testing platform of an Hexarotor UAV is completely realized. Measurements data issued from the onboard sensors are collected and a classification model to detect damaged propellers and failed motors has been built. To simulate a motor fault condition, its effectiveness is reduced using an RC transmitter. Propeller damages are simulated by clipping the propellers gradually. Experimental results demonstrate that artificial neural networks (ANN) techniques outperform other methods in terms of classification accuracy and are shown to be effective at identifying the different types of damaged propellers or failed motors.

In this review we have surveyed the literature on the autonomous vehicle focusing on the decision-making architecture. The main contribution of this paper is to provide an up-to-date reference that tries to cover as much as possible this broad topic rapidly. Therefore, after analyzing the body of literature, we were able to present a detailed overview of the usual anatomy of the decision-making system, review the coverage of public reviews and surveys available around the topic, present the environment and vehicle representation, and showcase the literature through: trajectory planning, risk-uncertainty assessment, and mimicking human behaviour mainly using perception-based methods. We also classified the literature according to scenarios covered, action-space used, and whether or not they emphasize the time horizon or frequency or time-step.