Deep Reinforcement Learning-Based Multi-Panel Beam Management in Massive MIMO Systems: Algorithm Design and System-Level Simulation

To adapt to the complicated interference and the high dynamics of wireless circumstances, deep reinforcement learning (DRL) has been considered as a potential solution for beam management in the massive multiple-input and multiple-output (MIMO) systems. However, due to the extremely high dimensions of both action and state spaces, the existing DRL-based schemes are with high computation costs, and the practical performance is still unknown. To provide some insights, DRL-based beam management in the massive MIMO systems is studied in this paper. First, a DRL-based beam management scheme has been designed for beyond the fifth generation and the sixth generation (B5G/6G) systems, which can support the collaborative beam selections of multiple panels with low complexity and fast convergence. Second, a system-level simulation platform is developed to evaluate the performance of our proposed scheme in B5G/6G systems. Finally, the system-level simulation results are provided, which show that our proposed scheme can achieve much higher spectrum efficiency than the referred evaluation results given by international telecommunication union (ITU).