Social Relationship Recognition Based on A Hybrid Deep Neural Network

Social relations reveal the interpersonal association of human beings. Developing techniques to automatically recognize social relations from visual data has great potential for improving human-computer interaction. In this paper, a hybrid deep network is proposed to predict the social relations between two human beings in an image. Unlike existing methods that typically learn deep learning models from scratch, a VGG-FACE model previously trained for face recognition is fine-tuned on a social relation database and used as branches of a siamese-like network. Moreover, a deep network is proposed to extract scene features that contain high-level information related to social relations from whole images and its predictions are fused with the predictions of the siamese network to generate the final result. Experiments show that the proposed approach saves the effort of pre-training and preparing auxiliary datasets, i.e. facial attribute datasets, and outperforms state-of-the-art methods.