MSDD: A Multimodal Language Dateset for Stance Detection

Stance Detection is the task of automatically determining whether the author of a text is positive, negative, or neutral towards a given target. Correct detecting stance is conducive to false news detection, claim validation, and argument search. Detecting stance from certain types of conversation, especially multimodal conversation is an interesting problem which has not been carefully explored. In social interaction, people usually express their stance on instance, which is produced in a multimodal manner, through the usage of words (text), gestures (video) and prosodic cues (audio). Stance detection is an established research area in NLP, but in a multimodal context it is an understudied area. In this paper, we present MSDD, a novel multimodal dataset for stance detection, to explore multimodal language for expressing stance in conversation. We conducted a series of experiments on MSDD, and the result shows that multimodal information indeed improves the dialogue stance detection to some extent, but the fusion of the multimodal language needs to be enhanced.