Delay-Optimal Random Access For Massive Heterogeneous Iot Devices

The random access (RA) decision of a device should depend on both its own state and the influence of others for avoiding the overload. However, the heterogeneous characteristics of massive devices lead to the difficult for estimating the influence. In this paper, we consider the RA problem for the large-scale energy harvesting IoT networks. To deal with the overload issue for massive heterogeneous devices, we propose a delay-optimal RA strategy by improving the conventional mean field games (MFG) via exchanging the mean field terms (MFT) among devices. Specifically, we formulate the delay-optimal problem as a two-dimensional Markov decision process (MDP) problem. For distributed deployment of massive random access, we divide the MDP into multiple per-device subproblems. With the given influence of other devices, i.e., MFT, we solve the per-device MDP and propose the optimal RA scheme via Hamilton-Jacobi-Bellman (HJB) equation. To obtain optimal access strategy, we adopt an online learning scheme to estimate the influence from others, where we transform MFT into the number of simultaneous access devices in order to reduce the state space significantly. Considering that the heterogeneous devices will cause the deviation of MFT estimation, we design a consensus scheme for the MFT based on stochastic approximation by information exchange among neighbor devices. Finally, simulation results show that the proposed RA scheme achieves a good delay performance comparing with other baselines.