Encoder-Decoder Reinforcement Learning for Active Search and Coverage in Point Clouds

Consider an active target search and coverage problem in which a mobile 3D sensor aims both to quickly find relevant targets in its environment and to quickly scan these targets so as to cover them. We propose a deep reinforcement learning method to find a sequence of sensor poses that solves this problem. The method uses a deep hierarchical encoder-decoder architecture that heavily modifies the point cloud transformer network. To reduce computation, a new way is proposed to select representative points from the cloud, based on cosine similarities between feature vectors. In simulated experiments, the novel encoder-decoder architecture significantly improves performance compared both to a greedy baseline, and to a network structure closer to the point cloud transformer. The architecture also improves classification and segmentation performance in supervised learning on point clouds. We successfully train much deeper networks than those usually employed in reinforcement learning, and we believe the ideas used to achieve this can be adapted to deep reinforcement learning in general.