Intelligent AUV Surfacing Control in Network Attack Scenario

Autonomous Underwater Vehicle (AUV) have become an important tool for humans to explore the ocean. The evolution of AUV is trending towards clustering. At present, research on the intelligent control strategy of AUV mostly focuses on the path following and trajectory tracking. Meanwhile, few studies concentrate on the control of an AUV in emergency situations. In the scenario of network attack, an emergency surfacing is necessary. In case of emergency, the loss of AUV can be reduced by timely and stable surfacing. The superiority of reinforcement learning (RL) in AUV control has been proven by many studies. In this work, we used an improved deep reinforcement learning (DRL) method based on deep deterministic policy gradient (DDPG) to solve the problem of AUV surfacing control in emergency situations. The method introduces expert experience data for pre-training and changes the update mechanism of the actor network, thereby improving the convergence rate and stability of the algorithm. Furthermore, we simulate the AUV surfacing control in five typical emergency situations, including partial thruster damage and rudder-jamming. Experimental results indicate that our method enables AUV to choose shorter paths in emergency situations, surfacing to the target area. Moreover, the method converges and maintains a stable reward function after training up to 900 episodes, showing faster convergence and stable performance.