Deep Reinforcement Learning with Priority Map Bias for Autonomous Cleaning-Audit Robots

Autonomous cleaning auditing requires effective path planning to gather representative dirt samples over large areas. This paper presents a novel audit-path planning method using deep reinforcement learning (DRL) guided by priority maps. A 2D priority map is constructed from the spatial distribution of objects likely to contribute to dirt accumulation and provided as input to a DRL agent that learns to select motion primitives and sampling actions. Three DRL algorithms, namely A3C, PPO, and MARWIL, are evaluated in simulation with and without priority map guidance. Results show that incorporating the priority map improves learning efficiency, accelerates convergence, and yields better audit paths in high-priority regions. The trained policies are validated on BELUGA, an in-house developed audit robot, in a real indoor environment. The MARWIL-based policy achieved higher dirt sample collection per unit distance than PPO, confirming the practical benefit of priority map guidance for autonomous cleaning audits.