DPOD: Domain-Specific Prompt Tuning for Multimodal Fake News Detection
arxiv(2023)
摘要
The spread of fake news using out-of-context images has become widespread and
is a relevant problem in this era of information overload. Such out-of-context
fake news may arise across different domains like politics, sports,
entertainment, etc. In practical scenarios, an inherent problem of imbalance
exists among news articles from such widely varying domains, resulting in a few
domains with abundant data, while the rest containing very limited data. Under
such circumstances, it is imperative to develop methods which can work in such
varying amounts of data setting. In this work, we explore whether out-of-domain
data can help to improve out-of-context misinformation detection (termed here
as multi-modal fake news detection) of a desired domain, to address this
challenging problem. Towards this goal, we propose a novel framework termed
DPOD (Domain-specific Prompt-tuning using Out-of-Domain data). First, to
compute generalizable features, we modify the Vision-Language Model, CLIP to
extract features that helps to align the representations of the images and
corresponding text captions of both the in-domain and out-of-domain data in a
label-aware manner. Further, we propose a domain-specific prompt learning
technique which leverages the training samples of all the available domains
based on the extent they can be useful to the desired domain. Extensive
experiments on a large-scale benchmark dataset, namely NewsCLIPpings
demonstrate that the proposed framework achieves state of-the-art performance,
significantly surpassing the existing approaches for this challenging task.
Code will be released on acceptance.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要