Multi-Scale Enhanced Active Learning for Skeleton-Based Action Recognition

Skeleton-based models have been widely used, because of their robustness to complex backgrounds and high computational efficiency. However, annotating skeleton sequences is labor-intensive. It is appealing to reduce the cost of acquiring data with accurate labels for skeleton-based models. This paper presents an active learning method for the skeleton-based action recognition model, which boosts the performance of the model with less labeled data by instructing humans to annotate the most valuable samples. The key issue in active learning is to train a model that precisely predicts the values of samples. To achieve this, we propose to enhance the ability of our model to evaluate samples by modeling actions from different granularities from multi-scale representations of skeletons. The multi-scale method is simple and easy to use, which can be treated as a plug-and-play extension to strengthen the skeleton-based models. We conduct experiments on the SHREC, NTU-60, and Kinetics-Skeleton. Extensive experimental results demonstrate the effectiveness of the proposed method.