QoE-aware Q-learning based approach to dynamic TDD uplink-downlink reconfiguration in indoor small cell networks

The continuing rise of the amount of mobile traffic is daunting, but the deploying of indoor small cells provides exciting opportunities to boost network capacity, extend cell coverage, and eventually thrive on an increased level of customers’ quality of experience (QoE). Unfortunately, in current wireless systems, traffic exhibits great variations in uplink and downlink directions, which introduces challenges of efficient resource allocation. Through using a dynamic time-division duplexing (TDD) method, network operators can flexibly adapt to such variations. However, cross-link interference appears in a dynamic TDD network and seriously suppresses uplink transmission. In this work, we proposed a decentralized QoE-aware reinforcement learning based approach to dynamic TDD reconfiguration. The objective is to maximize the utility function of the users’ QoE in an indoor small cell network. This was done by empowering each base station to select the best configuration to avoid the occurrence of cross-link interference while maintaining as many users that can enjoy their service at a satisfactory QoE as possible. At each episode, after collecting local reports of the QoE state and traffic load of the users, every base station dynamically chooses the best configuration according to the learning model. The learning process repeats itself until convergence. We implemented a simulator to evaluate the performances of the proposed algorithms. The results show that the proposed strategy achieves the best utility of QoE in comparison with other approaches, especially in the direction of the uplink transmission. The study demonstrates the great potential of harnessing reinforcement learning algorithms to attain higher QoE in small cell networks.