Self-Supervised Embodied Learning for Semantic Segmentation

Embodied learning methods enhance perception ability using observations. The existing techniques can be divided into two categories: investigating how to gather samples to improve the perceptual ability as much as possible and improving the perceptual ability with less human cost by self-supervised mechanism. For an embodied learning task, it is important to use fewer human costs to improve the perception ability as much as possible. We design the FC(Familiar-Curious) reward based on the consistency of the current and previous frames, and the policy based on this reward encourages the agent to explore the region with curious and familiar samples. We trained a vision-based agent to verify the effectiveness of our method. Experimental results demonstrate that our approach can improve perceptual ability without manual annotations.