Behavior recognition algorithm based on the improved R3D and LSTM network fusion

Because behavior recognition is based on video frame sequences,this paper proposes a behavior recognition algorithm that combines 3D residual convolutional neural network (R3D) and long short-term memory (LSTM).First,the residual module is extended to three dimensions,which can ex-tract features in the time and space domain at the same time.Second,by changing the size of the pooling layer window the integrity of the time domain features is preserved,at the same time,in or-der to overcome the difficulty of network training and over-fitting problems,the batch normalization(BN) layer and the dropout layer are added.After that,because the global average pooling layer(GAP) is affected by the size of the feature map,the network cannot be further deepened,so the convolution layer and maxpool layer are added to the R3D network.Finally,because LSTM has the ability to memorize information and can extract more abstract timing features,the LSTM network is introduced into the R3D network.Experimental results show that the R3D + LSTM network achieves 91％ recognition rate on the UCF-101 dataset.