3D Deployment and Energy Efficiency Optimization Based on DRL for RIS-assisted Air-to-Ground Communications Networks

Research indicates that replacing relays with the reconfigurable intelligent surface (RIS) can effectively reduce energy consumption under certain conditions. Therefore, introducing RIS into unmanned aerial vehicle (UAV) assisted airto- ground communication networks can further enhance communication performance. In this paper, we propose a RISUAV- assisted communication network scenario that utilizes the zero-forcing (ZF) precoding method to eliminate multi-user interference, and then optimize the BS transmit power by using the Dinkelbach algorithm. To address the optimization problem of bandwidth allocation, RIS phase shifts, and threedimensional (3D) coordinates of the RIS-UAV, we propose two deep reinforcement learning (DRL) algorithms, which are termed D3QN-MM and D3QN-Pure, respectively. Both D3QN-MM and D3QN-Pure utilize the dueling double deep Q-network (D3QN) for optimizing bandwidth allocation and the 3D coordinates of the RIS-UAV. However, D3QN-MM employs the traditional majorizeminimization (MM) algorithm for RIS phase shifts optimization, while D3QN-Pure utilizes the D3QN. By comparing them with other algorithms, such as the DRL algorithm, the advantages of the algorithms proposed in this paper are highlighted. Furthermore, compared to the amplify-and-forward (AF) relay, the RIS can achieve a 48% energy efficiency improvement. Besides, the D3QN-Pure algorithm provided up to 14.3% energy effciency improvement compared to the D3QN-MM algorithm.