IA-FPN: Interactive Aggregation Feature Pyramid Network for Action Detection

Action detection (Spatio-temporal action localization) is a task of locating the actors in every frame of the video and classifying their actions. The difficulty of action detection is how to interactively aggregate features to effectively extract spatio-temporal information. We propose the Interactive Aggregation Feature Pyramid Network (IA-FPN), which can well integrate 2D convolution features and 3D convolution features. It is divided into two steps: first aggregate 2D convolution features and 3D convolution features respectively, and then further aggregate the fused features at different resolutions. Besides, we improve the positive and negative sample allocation strategy based on yolov3, which can effectively improve the convergence speed of action detection network. The proposed algorithm's Frame-mAP can achieve 84.5% and 87.0% on benchmark datasets of J-HMDB-21 and UCF101-24 respectively, compared with the most state-of-the-art algorithms in recent years, our method achieves competitive performance.