Real-Time Multiple Dyadic Interaction Detection in Surveillance Videos in the Wild

Social distancing measures are proposed as the primary strategy to curb the spread of pandemics caused by respiratory pathogens. Therefore, non-intrusive techniques to identify human-human interactions in public spaces play an important role in the curtailment of disease spread. This paper proposes a novel computer vision-based system that identifies multiple co-occurring dyadic (two-person) interactions in a crowded scenario and classifies them into six action classes. Human skeletons are extracted from the RGB videos to eliminate the background noise improving the overall detection accuracy. Two approaches have been used for the classifier: X3D-M and X3D-M+Attention. The latter achieves higher classification accuracy due to the ability of the attention layer to capture the long-distance interdependence of video frames. The efficacy of the proposed model was evaluated across two different datasets on more than 5000 frames, thus enabling a robust detection model in different environments. The proposed model is the first dyadic interaction detector in the wild, which enables it to be used in public spaces and thereby identify and mitigate transmission of respiratory diseases.