Spatial Pyramid Attention Based Two-Stream Network for Fish Behavior Recognition

In intensive aquaculture, achieving accurate monitoring of fish behavior is crucial for improving efficiency. Previous studies on fish behavior recognition have been limited to recognizing fish school behavior from a global perspective, disregarding the importance of recognizing behaviors that occur in local regions. In this study, a spatial pyramid attention based two-stream network (SPA-TSN) is proposed to recognize fish behavior from both global and local perspective. SPA-TSN is comprised of a spatial stream and a motion stream, both of which use a spatial pyramid attention module to obtain the spatial and temporal features of fish behavior respectively. Specifically, the spatial pyramid attention module aggregates the features from different levels of the baseline to capture the correlation between local and global information. Then, a fusion feature is obtained by adding the temporal feature to the spatial feature. The three features are assigned different weights through a learnable multiplier $M$ for behavior recognition. To validate the effectiveness of the proposed method, experiments are conducted on our established fish behavior dataset which consists four common fish behaviors. The results demonstrate that the proposed method has the best performance with an accuracy of 95.904%, performing better than state-of-the-art methods.