Meta Soft Actor-Critic Based Robust Sequential Power Control in Vehicular Networks

Reinforcement learning has been widely used to train a sequential power control policy from simulation environment in Internet of Vehicles. However, disturbance is usually inevitably introduced when the agent getting into the practical environment from the simulation environment, which leads to a critical challenge when multiple links share a common spectrum. This paper is dedicated to addressing this issue in a two-pronged way. On one hand, we set the aim of policy learning as minimizing the network transmission outage probability from a risk-sensitive perspective. On the other hand, we propose a meta soft actor-critic based power control scheme, where the key hyperparameter is auto-adjusted to adapt to environment variations and L2 regulation is taken in the loss function of critic network to avoid overfitting to the simulation environment. Simulation results show that, the proposed algorithm achieves a higher successful transmission probability under the same conditions and is more robust under disturbed environment, than the baseline schemes.