Fusion of Transformer Model and Skeleton Detection Model for Abnormal Human Activity Detection with Transfer Learning

Abnormal activity detection is an area of study that focuses on finding abnormal activities like theft, vandalism, abuse, etc in video and CCTV footage. In this paper, a combination of skeleton based and transformer-based approach is used. The skeletons are used to enrich the motion of human bodies in the video. The transformer is used to model dynamic convolutions which helps to overcome the problems associated with fixed size kernels. To preserve the background information the architecture also uses feature extraction. The methodology used in the paper, also proposed a way to use transfer learning in the transformer network. The proposed framework provides excellent results outperforming many of existing methodologies. ROC (AUC score) attained by the proposed method for some classes of actions are up to 97.71%, which is even higher than current state of art approaches. The average ROC (AUC score) of our method is better than many of the approaches which are considered in the paper.