Where Are the Facts? Searching for Fact checked Information to Alleviate the Spread of Fake News

EMNLP 2020, 2020.

Cited by: 0|Bibtex|Views27
Other Links: arxiv.org
Keywords:
convolution neural networksContextual Text Matchingattention networkrelevance matchingMultimodal Attention NetworkMore(9+)
Weibo:
Our framework can be used for other multimodal retrieval tasks

Abstract:

Although many fact-checking systems have been developed in academia and industry, fake news is still proliferating on social media. These systems mostly focus on fact-checking but usually neglect online users who are the main drivers of the spread of misinformation. How can we use fact-checked information to improve users' consciousness...More

Code:

Data:

0
Introduction
  • The rampant spread of biased news, partisan stories, false claims and misleading information has raised heightened societal concerns in recent years.
  • Many reports pointed out that fabricated stories possibly caused citizens’ misperception about political candidates (Allcott and Gentzkow, 2017), manipulated stock prices (Kogan et al, 2019) and threatened public health (Ashoka, 2020; Alluri, 2019).
  • The proliferation of misinformation has provoked the rise of fact-checking systems worldwide.
  • Since 2014, the number of fact-checking outlets has totally increased 400% in 60 countries (Stencel, 2019).
  • Fabricated stories and hoaxes are still pervading the cyberspace.
Highlights
  • The rampant spread of biased news, partisan stories, false claims and misleading information has raised heightened societal concerns in recent years
  • Our Visual Matching Network (VMN) amazingly outperforms text-based ranking baselines in Snopes perhaps because fauxtography is one of the most popular categories on Snopes (Friggeri et al, 2014) while Politifact mainly fact-checks political claims. By using both text and images, our Multimodal Attention Network (MAN) shows an average increase of 17.2% over the best baselines with the maximum improvement of 39.6%
  • When tweets do not have any images, we can use Contextual Text Matching (CTM) which may find less relevant articles compared with MAN
  • We built our best model (MAN-A) on the full dataset but observed some reduction in NDCG@1 and NDCG@3, but not HIT@3 compared with results of SC2 on separate datasets maybe because of the false negatives described in Section 4
  • We observe that there may be false negatives in the full dataset, meaning that a FC-article fact-checks an original tweet but the article is viewed as an irrelevant one (i.e., 100% precision but less than 100% recall) because the FC-article was not embedded in a fact-checker’s reply
  • Our framework can be used for other multimodal retrieval tasks
Methods
Results
  • The authors adopt NDCG@K (Xiong et al, 2017) and HIT@K (He et al, 2017) as evaluation metrics.
  • The authors report mean HIT@K and NDCG@K where K ∈ [1, 3, 5] based on all queries.
  • Since over 99.5% queries have only one relevant document, HIT@K is almost equal to Recall@K.
  • HIT@1 is equal to NDCG@1.
  • Performance of the Basic Retrieval.
  • The authors test BM25 in three cases shown in Fig. 3: (1) queries are tweets’ text (BM25-T), (2) queries are text in tweets’ images (BM25-I) and (3) queries are tweets’ text + text in tweets’ images (BM25-TI)
Conclusion
  • Since Snopes and Politifact are the most popular fact-checking sites, building two models for them is an acceptable cost.
  • When tweets do not have any images, the authors can use CTM which may find less relevant articles compared with MAN.
  • By searching for FC-articles and incorporating fact-checked information into social media posts, the authors can warn users about fake news and discourage them from spreading misinformation.
  • The authors' framework uses text and images to search for FC-articles, achieving an average increase of 4.8% over best baselines with the maximum improvement of 11.2%.
  • The authors' framework can be used for other multimodal retrieval tasks
Summary
  • Introduction:

    The rampant spread of biased news, partisan stories, false claims and misleading information has raised heightened societal concerns in recent years.
  • Many reports pointed out that fabricated stories possibly caused citizens’ misperception about political candidates (Allcott and Gentzkow, 2017), manipulated stock prices (Kogan et al, 2019) and threatened public health (Ashoka, 2020; Alluri, 2019).
  • The proliferation of misinformation has provoked the rise of fact-checking systems worldwide.
  • Since 2014, the number of fact-checking outlets has totally increased 400% in 60 countries (Stencel, 2019).
  • Fabricated stories and hoaxes are still pervading the cyberspace.
  • Objectives:

    Observing the downside of existing methods and impacts of broadcasting verified news, the goal is to search for fact-checking articles (FC-articles) which address the content of original tweets.
  • The authors' paper deviates from these work since the authors aim to find FC-articles given multimodal fake news in social media posts.
  • As the goal is to increase users’ awareness of verified news, studies about fact-checkers (Vo and Lee, 2018, 2019; You et al, 2019) are close to ours.
  • Given an original tweet q and a FC-article d, where every original tweet q contains text and images and the article d contains text and/or images, the authors aim to derive function f (q, d) which determines their relevancy1
  • Methods:

    6.1 Neural Ranking Baselines

    The authors compare with 9 state-of-the-art neural ranking baselines, divided into 3 groups as follows: (1) multimodal retrieval methods including DVSH (Cao et al, 2016) and TranSearch (Guo et al, 2018), (2) semantic matching models including ESIM (Chen et al, 2017) and NSMN (Nie et al, 2019), and (3) relevance matching methods including MatchPyramid (Pang et al, 2016), KNRM (Xiong et al, 2017), ConvKNRM (Dai et al, 2018), CoPACRR (Hui et al, 2018) and DUET (Mitra et al, 2017).
  • Results:

    The authors adopt NDCG@K (Xiong et al, 2017) and HIT@K (He et al, 2017) as evaluation metrics.
  • The authors report mean HIT@K and NDCG@K where K ∈ [1, 3, 5] based on all queries.
  • Since over 99.5% queries have only one relevant document, HIT@K is almost equal to Recall@K.
  • HIT@1 is equal to NDCG@1.
  • Performance of the Basic Retrieval.
  • The authors test BM25 in three cases shown in Fig. 3: (1) queries are tweets’ text (BM25-T), (2) queries are text in tweets’ images (BM25-I) and (3) queries are tweets’ text + text in tweets’ images (BM25-TI)
  • Conclusion:

    Since Snopes and Politifact are the most popular fact-checking sites, building two models for them is an acceptable cost.
  • When tweets do not have any images, the authors can use CTM which may find less relevant articles compared with MAN.
  • By searching for FC-articles and incorporating fact-checked information into social media posts, the authors can warn users about fake news and discourage them from spreading misinformation.
  • The authors' framework uses text and images to search for FC-articles, achieving an average increase of 4.8% over best baselines with the maximum improvement of 11.2%.
  • The authors' framework can be used for other multimodal retrieval tasks
Tables
  • Table1: Split datasets
  • Table2: Performance of our models and baselines when using images and text in tweets
  • Table3: Performance of our models and baselines when using images, text in tweets and text in images
  • Table4: Ranking performances on leftover queries when using images, text in tweets and text in images
  • Table5: Effects of contextual word embeddings
Download tables as Excel
Related work
  • Fake News and Fact-checking. Fake news detection methods mainly use linguistics and textual content (Zellers et al, 2019; Zhao et al, 2015; Wang, 2017; Shu et al, 2019), temporal spreading patterns (Liu and Wu, 2018; Ma et al, 2018), network structures (Wu and Liu, 2018; Liu et al, 2020) and users’ feedbacks (Vo and Lee, 2019, 2020; Shu et al, 2019). Studies about multimodal fake news detection (Gupta et al, 2013; Wang et al, 2018b) are different from ours since their inputs are text and images of tweets while our inputs are pairs of a multimodal tweet and a FC-article.

    Our work is closely related to evidence-aware fact-checking. Thorne et al (2018); Nie et al (2019) built a pipeline to find documents and sentences to fact-check mutated claims generated from Wikipedia pages, Wang et al (2018a) aimed to find webpages related to given FC-articles and predict their stances on claims in the FC-articles. Popat et al (2018) only focused on fact-checking and (Shaar et al, 2020) detected previously factchecked claims. Our paper deviates from these work since we aim to find FC-articles given multimodal fake news in social media posts. As our goal is to increase users’ awareness of verified news, studies about fact-checkers (Vo and Lee, 2018, 2019; You et al, 2019) are close to ours.
Funding
  • This work was supported in part by NSF grant CNS-1755536, AWS Cloud Credits for Research, and Google Cloud
Study subjects and analysis
unique original tweets: 18961
We also only kept original tweets where text and images are both available. After preprocessing, we have 19,341 original tweet in English and FC-article pairs (q, d) in which there are 18,961 unique original tweets and 2,845 FC-articles. Following Vosoughi et al (2018), a labeling step is conducted to ensure that in each pair, the article fact-checks the original tweet

positive pairs: 13239
As we utilized the dataset in (Vo and Lee, 2019) which was collected during the 2016 U.S presidential election, many tweets and FC-articles were about misinformation related to Hillary Clinton and Donald Trump, leading to topically similar pairs which might confuse labelers. After labeling, we have a full dataset of 13,239 positive pairs made by 13,091 original tweets and 2,170 FC-articles. We observe that there may be false negatives in the full dataset, meaning that a FC-article actually fact-checks an original tweet but the article is viewed as an irrelevant one (i.e., 100% precision but less than 100% recall) because the FC-article was not embedded in a fact-checker’s reply

positive pairs: 11202
But the number of false negatives under this case would be smaller than those in the full dataset. In Snopes dataset, we have 11,202 positive pairs made by 11,167 tweets and 1,703 FC-articles. In PolitiFact dataset, we have 2,037 positive pairs made by 2,026 tweets and 467 FC-articles

positive pairs: 2037
In Snopes dataset, we have 11,202 positive pairs made by 11,167 tweets and 1,703 FC-articles. In PolitiFact dataset, we have 2,037 positive pairs made by 2,026 tweets and 467 FC-articles. There are 102 overlapping tweets between the two datasets

overlapping tweets: 102
In PolitiFact dataset, we have 2,037 positive pairs made by 2,026 tweets and 467 FC-articles. There are 102 overlapping tweets between the two datasets. The number of unique original posters is 8,277 and 1,482 in Snopes and Politifact respectively

topics of tweets: 5
Since the topic of an original tweet is related to the topic of a corresponding FC-article, we extracted topics of relevant FC-articles to understand the topical distribution of tweets. By analyzing each FC-article, top 5 topics of tweets in Snopes are as follows: Politics (42.3%), Fauxtography (22.7%), Junk News (8.1%), Uncategorized (6.8%), Quotes (4.8%). For Politifact, tweets’ topics are mostly about politics due to its political mission

cases: 3
Performance of the Basic Retrieval. We test BM25 in three cases shown in Fig. 3: (1) queries are tweets’ text (BM25-T), (2) queries are text in tweets’ images (BM25-I) and (3) queries are tweets’ text + text in tweets’ images (BM25-TI). In Fig. 3(a), HIT@50 of BM25-T is only 50% while BM25-I’s HIT@50 is 70%, suggesting that a lot of fake news appear in images

tweets: 1164
In conclusion, our model MAN outperforms all baselines in both two testing scenarios. Experiments on the leftover original tweets (i.e., 1,164 tweets in Snopes and 156 tweets in Politifact). We further test benefits of using text and images on each leftover query where we rank its x

original tweets: 910
We measure how much impact we can make on online users when correct FC-articles are retrieved (i.e. HIT@1 = 1). Totally, our best model, MAN-A, accurately finds FC-articles for 910 original tweets in test set of Snopes dataset. From these tweets, the total number of their retweets is 527,299 and total number of followers of the original posters who posted 910 original tweets is 233M

original tweets: 910
Totally, our best model, MAN-A, accurately finds FC-articles for 910 original tweets in test set of Snopes dataset. From these tweets, the total number of their retweets is 527,299 and total number of followers of the original posters who posted 910 original tweets is 233M. Roughly speaking, we can inform fact-checked information to millions of users

Reference
  • Hunt Allcott and Matthew Gentzkow. 2017. Social media and fake news in the 2016 election. Journal of economic perspectives, 31(2):211–36.
    Google ScholarLocate open access versionFindings
  • Aparna Alluri. 2019. Whatsapp: The ’black hole’ of fake news in india’s election. https://www.bbc.com/news/world-asia-india-47797151.
    Findings
  • Ashoka. 2020. Misinformation spreads faster than coronavirus: How a social organization in turkey is fighting fake news. https://bit.ly/36qqmmH.
    Findings
  • Saeid Balaneshin-kordan and Alexander Kotov. 2018. Deep neural architecture for multi-modal retrieval based on joint embedding space for text and images. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pages 28–36.
    Google ScholarLocate open access versionFindings
  • Yue Cao, Mingsheng Long, Jianmin Wang, Qiang Yang, and Philip S Yu. 2016. Deep visual-semantic hashing for cross-modal retrieval. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1445–1454.
    Google ScholarLocate open access versionFindings
  • Haolan Chen, Fred X Han, Di Niu, Dong Liu, Kunfeng Lai, Chenglin Wu, and Yu Xu. 2018. Mix: Multi-channel information crossing for text matching. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 110–119.
    Google ScholarLocate open access versionFindings
  • Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 201Enhanced lstm for natural language inference. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1657–1668.
    Google ScholarLocate open access versionFindings
  • Tao Chen, Xiangnan He, and Min-Yen Kan. 2016. Context-aware image tweet modelling and recommendation. In Proceedings of the 24th ACM international conference on Multimedia, pages 1018–1027.
    Google ScholarLocate open access versionFindings
  • CNN. 2020. How facebook is combating spread of covid-19 misinformation. https://cnn.it/3gjtBkg.
    Findings
  • Zhuyun Dai, Chenyan Xiong, Jamie Callan, and Zhiyuan Liu. 2018. Convolutional neural networks for soft-matching n-grams in ad-hoc search. In Proceedings of the eleventh ACM international conference on web search and data mining, pages 126– 134.
    Google ScholarLocate open access versionFindings
  • David Emery. 2016. President obama confirms he will refuse to leave office if trump is elected. https://bit.ly/36tbWCp.
    Findings
  • Dan Evon. 2017. Antifa member photographed beating police officer? https://bit.ly/3lcj2RL.
    Findings
  • Adrien Friggeri, Lada Adamic, Dean Eckles, and Justin Cheng. 2014. Rumor cascades. In Eighth International AAAI Conference on Weblogs and Social Media.
    Google ScholarLocate open access versionFindings
  • Jiafeng Guo, Yixing Fan, Qingyao Ai, and W Bruce Croft. 2016. A deep relevance matching model for ad-hoc retrieval. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pages 55–64.
    Google ScholarLocate open access versionFindings
  • Jiafeng Guo, Yixing Fan, Xiang Ji, and Xueqi Cheng. 2019. Matchzoo: A learning, practicing, and developing system for neural text matching. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1297–1300.
    Google ScholarLocate open access versionFindings
  • Yangyang Guo, Zhiyong Cheng, Liqiang Nie, XinShun Xu, and Mohan Kankanhalli. 2018. Multimodal preference modeling for product search. In Proceedings of the 26th ACM international conference on Multimedia, pages 1865–1873.
    Google ScholarLocate open access versionFindings
  • Aditi Gupta, Hemank Lamba, Ponnurangam Kumaraguru, and Anupam Joshi. 2013. Faking sandy: characterizing and identifying fake images on twitter during hurricane sandy. In Proceedings of the 22nd international conference on World Wide Web, pages 729–736.
    Google ScholarLocate open access versionFindings
  • Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770– 778.
    Google ScholarLocate open access versionFindings
  • Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web, pages 173– 182.
    Google ScholarLocate open access versionFindings
  • Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. 2013. Learning deep structured semantic models for web search using clickthrough data. In Proceedings of the 22nd ACM international conference on Information & Knowledge Management, pages 2333–2338.
    Google ScholarLocate open access versionFindings
  • Kai Hui, Andrew Yates, Klaus Berberich, and Gerard De Melo. 2018. Co-pacrr: A context-aware neural ir model for ad-hoc retrieval. In Proceedings of the eleventh ACM international conference on web search and data mining, pages 279–287.
    Google ScholarLocate open access versionFindings
  • Kai Hui, Andrew Yates, Klaus Berberich, and Gerard de Melo. 2017. Pacrr: A position-aware neural ir model for relevance matching. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1049–1058.
    Google ScholarLocate open access versionFindings
  • Shan Jiang and Christo Wilson. 2018. Linguistic signals under misinformation and fact-checking: Evidence from user comments on social media. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW):1–23.
    Google ScholarLocate open access versionFindings
  • Jin-Hwa Kim, Jaehyun Jun, and Byoung-Tak Zhang. 2018. Bilinear attention networks. In Advances in Neural Information Processing Systems, pages 1564–1574.
    Google ScholarLocate open access versionFindings
  • Liang Pang, Yanyan Lan, Jiafeng Guo, Jun Xu, Shengxian Wan, and Xueqi Cheng. 2016. Text matching as image recognition. In Thirtieth AAAI Conference on Artificial Intelligence.
    Google ScholarLocate open access versionFindings
  • Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. In International Conference on Learning Representations.
    Google ScholarLocate open access versionFindings
  • Shimon Kogan, Tobias J Moskowitz, and Marina Niessner. 2019. Fake news: Evidence from financial markets. Available at SSRN 3237763.
    Google ScholarFindings
  • Katrien Laenen, Susana Zoghbi, and Marie-Francine Moens. 2018. Web search of fashion items with multimodal querying. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pages 342–350.
    Google ScholarLocate open access versionFindings
  • Stephan Lewandowsky, Ullrich KH Ecker, Colleen M Seifert, Norbert Schwarz, and John Cook. 2012. Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest, 13(3):106–131.
    Google ScholarLocate open access versionFindings
  • Yang Liu and Yi-Fang Brook Wu. 2018. Early detection of fake news on social media through propagation path classification with recurrent and convolutional networks. In Thirty-Second AAAI Conference on Artificial Intelligence.
    Google ScholarLocate open access versionFindings
  • Zhenghao Liu, Chenyan Xiong, and Maosong Sun. 2020. Fine-grained fact verification with kernel graph attention network.
    Google ScholarFindings
  • Jing Ma, Wei Gao, and Kam-Fai Wong. 2018. Rumor detection on twitter with tree-structured recursive neural networks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1980– 1989.
    Google ScholarLocate open access versionFindings
  • Ryan McDonald, George Brokos, and Ion Androutsopoulos. 2018. Deep relevance ranking using enhanced document-query interactions. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1849–1860.
    Google ScholarLocate open access versionFindings
  • Liang Pang, Yanyan Lan, Jiafeng Guo, Jun Xu, Jingfang Xu, and Xueqi Cheng. 2017. Deeprank: A new deep architecture for relevance ranking in information retrieval. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pages 257–266.
    Google ScholarLocate open access versionFindings
  • Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543.
    Google ScholarLocate open access versionFindings
  • Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of NAACL-HLT, pages 2227–2237.
    Google ScholarLocate open access versionFindings
  • Kashyap Popat, Subhabrata Mukherjee, Jannik Strotgen, and Gerhard Weikum. 2017. Where the truth lies: Explaining the credibility of emerging claims on the web and social media. In Proceedings of the 26th International Conference on World Wide Web Companion, pages 1003–1012. International World Wide Web Conferences Steering Committee.
    Google ScholarLocate open access versionFindings
  • Kashyap Popat, Subhabrata Mukherjee, Andrew Yates, and Gerhard Weikum. 2018. Declare: Debunking fake news and false claims using evidence-aware deep learning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 22–32.
    Google ScholarLocate open access versionFindings
  • Jinfeng Rao, Linqing Liu, Yi Tay, Wei Yang, Peng Shi, and Jimmy Lin. 2019a. Bridging the gap between relevance matching and semantic matching for short text similarity modeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5373–5384.
    Google ScholarLocate open access versionFindings
  • Bhaskar Mitra, Fernando Diaz, and Nick Craswell. 2017. Learning to match using local and distributed representations of text for web search. In Proceedings of the 26th International Conference on World Wide Web, pages 1291–1299.
    Google ScholarLocate open access versionFindings
  • Yixin Nie, Haonan Chen, and Mohit Bansal. 2019. Combining fact extraction and verification with neural semantic matching networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6859–6866.
    Google ScholarLocate open access versionFindings
  • Sara Ashley O’Brien. 2018. Deepfakes are coming. is big tech ready? https://cnnmon.ie/2AWCCix.
    Findings
  • OCR Space. 2020. Free ocr api and online ocr. https://ocr.space/.
    Findings
  • Jinfeng Rao, Wei Yang, Yuhao Zhang, Ferhan Ture, and Jimmy Lin. 2019b. Multi-perspective relevance matching with hierarchical convnets for social media search. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 232– 240.
    Google ScholarLocate open access versionFindings
  • Aliaksei Severyn and Alessandro Moschitti. 2015. Learning to rank short text pairs with convolutional deep neural networks. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pages 373– 382.
    Google ScholarLocate open access versionFindings
  • Shaden Shaar, Giovanni Da San Martino, Nikolay Babulkov, and Preslav Nakov. 2020. That is a known lie: Detecting previously fact-checked claims. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3607– 3618.
    Google ScholarLocate open access versionFindings
  • Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Gregoire Mesnil. 2014. A latent semantic model with convolutional-pooling structure for information retrieval. In Proceedings of the 23rd ACM international conference on conference on information and knowledge management, pages 101–110.
    Google ScholarLocate open access versionFindings
  • Kai Shu, Limeng Cui, Suhang Wang, Dongwon Lee, and Huan Liu. 2019. defend: Explainable fake news detection. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 395–405.
    Google ScholarLocate open access versionFindings
  • Mark Stencel. 2019. Number of fact-checking outlets surges to 188 in more than 60 countries. https://bit.ly/36y3S3l.
    Findings
  • Yi Tay, Anh Tuan Luu, Aston Zhang, Shuohang Wang, and Siu Cheung Hui. 2019. Compositional deattention networks. In Advances in Neural Information Processing Systems, pages 6132–6142.
    Google ScholarLocate open access versionFindings
  • James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. Fever: a large-scale dataset for fact extraction and verification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 809–819.
    Google ScholarLocate open access versionFindings
  • Anthony J Viera, Joanne M Garrett, et al. 2005. Understanding interobserver agreement: the kappa statistic. Fam med, 37(5):360–363.
    Google ScholarLocate open access versionFindings
  • Bokun Wang, Yang Yang, Xing Xu, Alan Hanjalic, and Heng Tao Shen. 2017. Adversarial cross-modal retrieval. In Proceedings of the 25th ACM international conference on Multimedia, pages 154–162.
    Google ScholarLocate open access versionFindings
  • William Yang Wang. 2017. “liar, liar pants on fire”: A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 422–426.
    Google ScholarLocate open access versionFindings
  • Xuezhi Wang, Cong Yu, Simon Baumgartner, and Flip Korn. 2018a. Relevant document discovery for factchecking articles. In Companion Proceedings of the The Web Conference 2018, pages 525–533.
    Google ScholarLocate open access versionFindings
  • Yaqing Wang, Fenglong Ma, Zhiwei Jin, Ye Yuan, Guangxu Xun, Kishlay Jha, Lu Su, and Jing Gao. 2018b. Eann: Event adversarial neural networks for multi-modal fake news detection. In Proceedings of the 24th acm sigkdd international conference on knowledge discovery & data mining, pages 849– 857.
    Google ScholarLocate open access versionFindings
  • Liang Wu and Huan Liu. 2018. Tracing fake-news footprints: Characterizing social media messages by how they propagate. In Proceedings of the eleventh ACM international conference on Web Search and Data Mining, pages 637–645.
    Google ScholarLocate open access versionFindings
  • Yu Wu, Wei Wu, Chen Xing, Ming Zhou, and Zhoujun Li. 2017. Sequential matching network: A new architecture for multi-turn response selection in retrieval-based chatbots. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 496–505.
    Google ScholarLocate open access versionFindings
  • Nguyen Vo and Kyumin Lee. 2018. The rise of guardians: Fact-checking url recommendation to combat fake news. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 275–284.
    Google ScholarLocate open access versionFindings
  • Nguyen Vo and Kyumin Lee. 2019. Learning from factcheckers: Analysis and generation of fact-checking language. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 335–344.
    Google ScholarLocate open access versionFindings
  • Nguyen Vo and Kyumin Lee. 2020. Standing on the shoulders of guardians: Novel methodologies to combat fake news. In Disinformation, Misinformation, and Fake News in Social Media, pages 183– 210. Springer.
    Google ScholarLocate open access versionFindings
  • Soroush Vosoughi, Deb Roy, and Sinan Aral. 2018. The spread of true and false news online. Science, 359(6380):1146–1151.
    Google ScholarLocate open access versionFindings
  • Chenyan Xiong, Zhuyun Dai, Jamie Callan, Zhiyuan Liu, and Russell Power. 2017. End-to-end neural ad-hoc ranking with kernel pooling. In Proceedings of the 40th International ACM SIGIR conference on research and development in information retrieval, pages 55–64.
    Google ScholarLocate open access versionFindings
  • Di You, Nguyen Vo, Kyumin Lee, and Qiang Liu. 2019. Attributed multi-relational attention network for fact-checking url recommendation. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 1471–1480.
    Google ScholarLocate open access versionFindings
  • Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoc V Le. 2018. Qanet: Combining local convolution with global self-attention for reading comprehension. In Proceedings of the 6th International Conference on Learning Representations.
    Google ScholarLocate open access versionFindings
  • Shengxian Wan, Yanyan Lan, Jiafeng Guo, Jun Xu, Liang Pang, and Xueqi Cheng. 2016. A deep architecture for semantic matching with multiple positional sentence representations. In Thirtieth AAAI Conference on Artificial Intelligence.
    Google ScholarLocate open access versionFindings
  • Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. In Advances in Neural Information Processing Systems, pages 9051–9062.
    Google ScholarLocate open access versionFindings
  • Zhe Zhao, Paul Resnick, and Qiaozhu Mei. 2015. Enquiring minds: Early detection of rumors in social media from enquiry posts. In Proceedings of the 24th international conference on world wide web, pages 1395–1405.
    Google ScholarLocate open access versionFindings
  • Ming Zhu, Aman Ahuja, Wei Wei, and Chandan K Reddy. 2019. A hierarchical attention retrieval model for healthcare question answering. In The World Wide Web Conference, pages 2472–2482.
    Google ScholarLocate open access versionFindings
  • Multimodal Retrieval Models. DVSH (Cao et al., 2016) accepts a pair of a multimodal query and a multimodal article, and outputs similarity score. It uses cosine max-margin loss. We also tried to compare with DVSH by using hashcode of queries’ text to match articles’ images and vice versa. However, DVSH did not perform well perhaps because queries’ text and documents’ images may be not semantically similar. We implemented DVSH by ourselves because there is no publicly downloadable code. We set its hidden size to 300 and used AlexNet to extract visual features by following (Cao et al., 2016).
    Google ScholarFindings
  • ELMo and multiple BiLSTM layers on our documents with 1,000 tokens. Relevance Matching Models. We compare with several state-of-the-art models in this category. MatchPyramid (Pang et al., 2016) uses CNN to capture spatial patterns. KNRM (Xiong et al., 2017) and ConvKNRM (Dai et al., 2018) use RBF kernel to pool n-gram matching signals. CoPACRR (Hui et al., 2018) uses similarities between queries’ representations and context-aware representations of words in documents to attend to matching signals. DUET (Mitra et al., 2017) unifies semantic and relevance matching signals into one model.
    Google ScholarLocate open access versionFindings
  • Implementation of MatchPyramid, KNRM, ConvKNRM, and DUET was obtained from MatchZoo (Guo et al., 2019). In MatchPyramid, we used default setting of MatchZoo. The number of kernels of KNRM and ConvKNRM was chosen from {7, 9, 11}. In ConvKNRM, we set |filters| to 300 like the word embeddings’ dimension size. n-gram was chosen from {1, 2, 3}. In DUET, we followed the same architecture proposed in (Mitra et al., 2017).
    Google ScholarLocate open access versionFindings
Full Text
Your rating :
0

 

Tags
Comments