Compound-Action DRL for Timely Data Collection in UAV-Assisted IoT with Stochastic Arrival.

The use of unmanned aerial vehicles (UAVs) in IoT networks can provide an efficient means of collecting mission-critical data. However, ensuring the freshness of collected data is a significant challenge due to the stochastic status updates of sensor nodes (SN s), In this paper, we delve into the issue of timely data collection aided by a UAV in IoT networks, where the UAV collects status updates from SNs and offloads the data to the data center at an opportune location. SNs sample information from their surroundings at stochastic intervals to generate new status updates. To minimize the average total age of information (AoI), we describe the problem as a Markov decision process and propose a compound-action deep reinforcement learning (CADRL) approach to jointly optimize the trajectory, scheduling of SNs, offloading strategy, and offloading power of the UAV, which can handle both continuous and discrete actions of the UAV. Simulation results indicate that our algorithm effectively reduces the AoI when contrasted with baseline methods.