When Crowdsourcing Meets Unmanned Vehicles: Toward Cost-Effective Collaborative Urban Sensing via Deep Reinforcement Learning

Mobile crowdsensing (MCS) and unmanned vehicle sensing (UVS) provide two complementary paradigms for large-scale urban sensing. Generally, MCS has a lower cost but often confronts sensing imbalance and even blind areas due to the limitation of human mobility, whereas UVS is often capable of completing more demanding tasks at the expense of limited energy supply and hardware cost. Thus, it is significant to investigate whether we could integrate the two paradigms for high-quality urban sensing in a cost-effective collaborative way. However, it is nontrivial due to complex and long-term optimization objectives, uncontrolled dynamics, and a large number of heterogeneous agents. To address the collaborative sensing problem, we propose an actor-critic-based heterogeneous collaborative reinforcement learning (HCRL) algorithm, which leverages several key ideas: local observation to handle expanded state space and extract the states of neighbor nodes, generalized model to avoid environment nonstationarity and ensure the scalability and stability of network, and proximal policy optimization to prevent the destructively large policy updates. Extensive simulations based on a mobility model and a realistic trace data set are conducted to confirm that HCRL outperforms the state-of-the-art baselines.