Hidden Enemy Visualization Using Fast Panoptic Segmentation on Battlefields

Most of enemies in battlefields are not visible due to cover and concealment, and it yields fear and a weakened combat power for allies. In order to overcome it, we present a new approach for visualizing hidden enemies in battlefields to enhance cognition ability and survival for soldiers. Our method is composed of two separate sub-task networks. One is an efficient real-time panoptic segmentation network based on YOLACT [1] to find hidden enemies as well as to understand scenes from the viewpoint of soldiers. The other is an image completion network to reconstruct occluded parts of enemies which is guided by the panoptic segmentation networks. Our experiments on the Cityscapes benchmarks show that the proposed panoptic segmentation network achieves almost realtime speed without significant performance drops. We also demonstrate qualitative results of our segmentation-guided image completion method successfully on a dataset constructed from images of the Battlefield4 game.