Social Behavior Atlas: A few-shot learning framework for multi-animal 3D social pose estimation, identification, and behavior embedding

The fact that multi-animal behavior quantification is still technically challenging nowadays greatly limits the accuracy and granularity of social behavior analysis. Data labeling of deep-learning-based approaches can be incredibly laborious, especially when multiple animals closely interact with each other, under which circumstances animal identity switching and body occlusions are likely to happen. To overcome the challenge, we designed a novel framework - Social Behavior Atlas (SBeA) and it shows promising results. SBeA utilizes a much smaller number of labeled frames for multi-animal 3D pose estimation, achieves label-free identification recognition, and successfully applies unsupervised dynamic learning for social behavior classification. Our results also demonstrate that SBeA is capable of achieving high performance across various species using existing customized datasets. Therefore, we can build a multi-animal behavior atlas. To start with, we construct a social behavior atlas for autism spectrum disorder (ASD) knockout mice, and SBeA reveals several behavioral biomarkers which were neglected before. These findings highlight the immense potential of SBeA for quantifying subtle social behaviors and provide a new paradigm in the fields of neuroscience and ecology.