A Spatial-Temporal Recurrent Neural Network for Video Saliency Prediction
IEEE Transactions on Image Processing(2021)
摘要
In this paper, a recurrent neural network is designed for video saliency prediction considering spatial-temporal features. In our work, video frames are routed through the static network for spatial features and the dynamic network for temporal features. For the spatial-temporal feature integration, a novel select and re-weight fusion model is proposed which can learn and adjust the fusion weights based on the spatial and temporal features in different scenes automatically. Finally, an attention-aware convolutional long short term memory (ConvLSTM) network is developed to predict salient regions based on the features extracted from consecutive frames and generate the ultimate saliency map for each video frame. The proposed method is compared with state-of-the-art saliency models on five public video saliency benchmark datasets. The experimental results demonstrate that our model can achieve advanced performance on video saliency prediction.
更多查看译文
关键词
Video saliency,spatial-temporal features,feature fusion,visual attention,deep learning
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络