That is a Known Lie: Detecting Previously Fact-Checked Claims

Shaden Shaar
Shaden Shaar
Nikolay Babulkov
Nikolay Babulkov

ACL, pp. 3607-3618, 2020.

Cited by: 0|Bibtex|Views29
EI
Other Links: dblp.uni-trier.de|academic.microsoft.com
Weibo:
We further report Mean Average Precision@k and HasPositive@k for k ∈ {10, 20} as well as Mean Average Precision, which would be more suitable in a non-real-time scenario, where recall would be more important

Abstract:

The recent proliferation of "fake news" has triggered a number of responses, most notably the emergence of several manual fact-checking initiatives. As a result and over time, a large number of fact-checked claims have been accumulated, which increases the likelihood that a new claim in social media or a new statement by a politician migh...More
0
Introduction
  • Governments, international organizations, tech companies, media, journalists, and regular users launched a number of initiatives to limit the impact of the newly emerging large-scale weaponization of disinformation1 online.
  • This included manual fact-checking initiatives, which aimed at debunking various false claims, with the hope to limit its impact, and to educate the public that not all claims online are true.
  • There have been datasets that combine claims from multiple fact-checking organizations (Augenstein et al, 2019), again with the aim of performing automatic fact-checking
Highlights
  • Governments, international organizations, tech companies, media, journalists, and regular users launched a number of initiatives to limit the impact of the newly emerging large-scale weaponization of disinformation1 online
  • This included manual fact-checking initiatives, which aimed at debunking various false claims, with the hope to limit its impact, and to educate the public that not all claims online are true
  • While some organizations debunked just a couple of hundred claims, others such as Politifact,3 FactCheck.org,4 Snopes,5 and Full Fact6 have fact-checked thousands or even tens of thousands of claims. The value of these collections of resources has been recognized in the research community, and they have been used to train systems to perform automatic fact-checking (Popat et al, 2017; Wang, 2017; Zlatkova et al, 2019) or to detect checkworthy claims in political debates (Hassan et al, 2015; Gencheva et al, 2017; Patwari et al, 2017; Vasileva et al, 2019)
  • We report HasPositive@k, i.e., whether there is a true positive among the top-k results. Measures such as Mean Average Precision@k and HasPositive@k for k ∈ {1, 3, 5} would be relevant in a scenario, where a journalist needs to verify claims in real time, in which case the system would return a short list of 3-5 claims that the journalist can quickly skim and make sure they are a true match
  • We further report Mean Average Precision@k and HasPositive@k for k ∈ {10, 20} as well as Mean Average Precision, which would be more suitable in a non-real-time scenario, where recall would be more important
Methods
  • The authors describe the experiments on the PolitiFact and the Snopes datasets. The authors start with IRbased models, followed by BERT-based semantic similarity on claims and articles, and the authors experiment with pairwise learning-to-rank models.

    7.1 Politifact Experiments

    For the PolitFact dataset, the authors perform experiments with all models from Section 6, and the authors report the results in Table 5.

    7.1.1 Experiment 1

    BM25-based Baselines

    The authors ran experiments matching the Input against Title, VerClaim, Body and Title+VerClaim+Body.
  • The authors can further see that the best representation, on all measures, is to use the Body, which performs better than using VerClaim by 0.12-0.14 in terms of MAP@k and MAP, and by 0.09 on MRR
  • This is probably because the article body is longer, which increases the probability of having more words matching the input claim.
  • The authors can see in Table 6 that, just like for PolitiFact, using VerClaim performed better than using the article title, which is true for all evaluation measures; this time the margin was much smaller than it was for PolitiFact.
  • BM25 is a very strong baseline for Snopes due to the high word overlap between the input claims and the verified claims
Results
  • Evaluation Measures

    The authors treat the task as a ranking problem. the authors use ranking evaluation measures, namely mean reciprocal rank (MRR), Mean Average Precision (MAP), and MAP truncated to rank k (MAP@k).
  • The authors report HasPositive@k, i.e., whether there is a true positive among the top-k results.
  • Measures such as MAP@k and HasPositive@k for k ∈ {1, 3, 5} would be relevant in a scenario, where a journalist needs to verify claims in real time, in which case the system would return a short list of 3-5 claims that the journalist can quickly skim and make sure they are a true match.
Conclusion
  • The authors have created specialized datasets, which the authors have released, together with the code, to the research community in order to enable further research.
  • The authors have presented learning-torank experiments, demonstrating sizable improvements over state-of-the-art retrieval and textual similarity approaches.
  • The authors plan to extend this work to more datasets and to more languages.
  • The authors further want to go beyond textual claims, and to take claimimage and claim-video pairs as an input
Summary
  • Introduction:

    Governments, international organizations, tech companies, media, journalists, and regular users launched a number of initiatives to limit the impact of the newly emerging large-scale weaponization of disinformation1 online.
  • This included manual fact-checking initiatives, which aimed at debunking various false claims, with the hope to limit its impact, and to educate the public that not all claims online are true.
  • There have been datasets that combine claims from multiple fact-checking organizations (Augenstein et al, 2019), again with the aim of performing automatic fact-checking
  • Methods:

    The authors describe the experiments on the PolitiFact and the Snopes datasets. The authors start with IRbased models, followed by BERT-based semantic similarity on claims and articles, and the authors experiment with pairwise learning-to-rank models.

    7.1 Politifact Experiments

    For the PolitFact dataset, the authors perform experiments with all models from Section 6, and the authors report the results in Table 5.

    7.1.1 Experiment 1

    BM25-based Baselines

    The authors ran experiments matching the Input against Title, VerClaim, Body and Title+VerClaim+Body.
  • The authors can further see that the best representation, on all measures, is to use the Body, which performs better than using VerClaim by 0.12-0.14 in terms of MAP@k and MAP, and by 0.09 on MRR
  • This is probably because the article body is longer, which increases the probability of having more words matching the input claim.
  • The authors can see in Table 6 that, just like for PolitiFact, using VerClaim performed better than using the article title, which is true for all evaluation measures; this time the margin was much smaller than it was for PolitiFact.
  • BM25 is a very strong baseline for Snopes due to the high word overlap between the input claims and the verified claims
  • Results:

    Evaluation Measures

    The authors treat the task as a ranking problem. the authors use ranking evaluation measures, namely mean reciprocal rank (MRR), Mean Average Precision (MAP), and MAP truncated to rank k (MAP@k).
  • The authors report HasPositive@k, i.e., whether there is a true positive among the top-k results.
  • Measures such as MAP@k and HasPositive@k for k ∈ {1, 3, 5} would be relevant in a scenario, where a journalist needs to verify claims in real time, in which case the system would return a short list of 3-5 claims that the journalist can quickly skim and make sure they are a true match.
  • Conclusion:

    The authors have created specialized datasets, which the authors have released, together with the code, to the research community in order to enable further research.
  • The authors have presented learning-torank experiments, demonstrating sizable improvements over state-of-the-art retrieval and textual similarity approaches.
  • The authors plan to extend this work to more datasets and to more languages.
  • The authors further want to go beyond textual claims, and to take claimimage and claim-video pairs as an input
Tables
  • Table1: PolitiFact: Input–verified claim pairs. The input claims are sentences from the 2016 US Presidential debates, and the verified claims are their corresponding fact-checked counter-parts in PolitiFact
  • Table2: Snopes: Input–VerClaim claim pairs. The input claims are tweets and the verified claims are their corresponding fact-checked counter-parts in Snopes
  • Table3: Statistics about the datasets: shown are the number of Input–VerClaim pairs and the total number of VerClaim claims to match an Input claim against. Note that each VerClaim comes with an associated factchecking analysis document in PolitiFact/Snopes
  • Table4: Analysis of the task complexity: number of Input–VerClaim pairs in PolitiFact and Snopes with TF.IDF-weighted cosine similarity above a threshold
  • Table5: PolitiFact: evaluation results on the test set
  • Table6: Snopes: evaluation results on the test set
Download tables as Excel
Related work
  • To the best of our knowledge, the task of detecting whether a claim has been previously fact-checked was not addressed before. Hassan et al (2017) mentioned it as an integral step of their end-to-end automated fact-checking pipeline, but there was very little detail provided about this component and it was not evaluated.

    In an industrial setting, Google has developed Fact Check Explorer,9 which is an exploration tool that allows users to search a number of factchecking websites (those that use ClaimReview from schema.org10) for the mentions of a topic, a person, etc. However, the tool cannot handle a complex claim, as it runs Google search, which is not optimized for semantic matching of long claims. While this might change in the future, as there have been reports that Google has started using BERT in its search, at the time of writing, the tool could not handle a long claim as an input.

    9http://toolbox.google.com/factcheck/ explorer

    10http://schema.org/ClaimReview

    A very similar work is the ClaimsKG dataset and system (Tchechmedjiev et al, 2019), which includes 28K claims from multiple sources, organized into a knowledge graph (KG). The system can perform data exploration, e.g., it can find all claims that contain a certain named entity or keyphrase. In contrast, we are interested in detecting whether a claim was previously fact-checked.
Reference
  • Zeynep Akkalyoncu Yilmaz, Shengjin Wang, Wei Yang, Haotian Zhang, and Jimmy Lin. 2019a. Applying BERT to document retrieval with Birch. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP ’19, pages 19– 24, Hong Kong, China.
    Google ScholarLocate open access versionFindings
  • Zeynep Akkalyoncu Yilmaz, Wei Yang, Haotian Zhang, and Jimmy Lin. 2019b. Cross-domain modeling of sentence-level evidence for document retrieval. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP ’19, pages 3488–3494, Hong Kong, China.
    Google ScholarLocate open access versionFindings
  • Pepa Atanasova, Lluıs Marquez, Alberto BarronCedeno, Tamer Elsayed, Reem Suwaileh, Wajdi Zaghouani, Spas Kyuchukov, Giovanni Da San Martino, and Preslav Nakov. 2018. Overview of the CLEF-2018 CheckThat! Lab on automatic identification and verification of political claims, Task 1: Check-worthiness. In Working Notes of the Conference and Labs of the Evaluation Forum, CLEF ’18, Avignon, France.
    Google ScholarLocate open access versionFindings
  • Pepa Atanasova, Preslav Nakov, Georgi Karadzhov, Mitra Mohtarami, and Giovanni Da San Martino. 2019. Overview of the CLEF-2019 CheckThat! Lab on Automatic Identification and Verification of Claims. Task 1: Check-Worthiness. In Working Notes of the Conference and Labs of the Evaluation Forum, CLEF ’19, Lugano, Switzerland.
    Google ScholarLocate open access versionFindings
  • Isabelle Augenstein, Christina Lioma, Dongsheng Wang, Lucas Chaves Lima, Casper Hansen, Christian Hansen, and Jakob Grue Simonsen. 2019. MultiFC: A real-world multi-domain dataset for evidence-based fact checking of claims. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP ’19, pages 4684–4696, Hong Kong, China.
    Google ScholarLocate open access versionFindings
  • Ramy Baly, Georgi Karadzhov, Dimitar Alexandrov, James Glass, and Preslav Nakov. 2018a. Predicting factuality of reporting and bias of news media sources. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP ’18, pages 3528–3539, Brussels, Belgium.
    Google ScholarLocate open access versionFindings
  • Alberto Barron-Cedeno, Tamer Elsayed, Giovanni Da San Martino, Maram Hasanain, Reem Suwaileh, Fatima Haouari, and Preslav Nakov. 2020. CheckThat! at CLEF 2020: Enabling the automatic identification and verification of claims on social media. In Proceedings of the European Conference on Information Retrieval, ECIR ’20, pages 499–507, Lisbon, Portugal.
    Google ScholarLocate open access versionFindings
  • Luisa Bentivogli, Ido Dagan, Hoa Trang Dang, Danilo Giampiccolo, and Bernardo Magnini. 2009. The fifth PASCAL recognizing textual entailment challenge. In Proceedings of the Text Analysis Conference, TAC ’09, Gaithersburg, MD, USA.
    Google ScholarLocate open access versionFindings
  • Kevin R. Canini, Bongwon Suh, and Peter L. Pirolli. 2011. Finding credible information sources in social networks based on content and social structure. In Proceedings of the IEEE International Conference on Privacy, Security, Risk, and Trust, and the IEEE International Conference on Social Computing, SocialCom/PASSAT ’11, pages 1–8, Boston, MA, USA.
    Google ScholarLocate open access versionFindings
  • Carlos Castillo, Marcelo Mendoza, and Barbara Poblete. 2011. Information credibility on Twitter. In Proceedings of the 20th International Conference on World Wide Web, WWW ’11, pages 675–684, Hyderabad, India.
    Google ScholarLocate open access versionFindings
  • Daniel Cer, Mona Diab, Eneko Agirre, Inigo LopezGazpio, and Lucia Specia. 2017. SemEval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation, SemEval ’17, pages 1–14, Vancouver, Canada.
    Google ScholarLocate open access versionFindings
  • Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Brian Strope, and Ray Kurzweil. 2018. Universal sentence encoder for English. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP ’18, pages 169–174, Brussels, Belgium.
    Google ScholarLocate open access versionFindings
  • Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. 2019. TabFact: A large-scale dataset for table-based fact verification.
    Google ScholarFindings
  • Giovanni Luca Ciampaglia, Prashant Shiralkar, Luis M. Rocha, Johan Bollen, Filippo Menczer, and Alessandro Flammini. 2015. Computational fact checking from knowledge networks. PLOS ONE, 10(6):1–13.
    Google ScholarLocate open access versionFindings
  • Ramy Baly, Mitra Mohtarami, James Glass, Lluıs Marquez, Alessandro Moschitti, and Preslav Nakov. 2018b. Integrating stance detection and fact checking in a unified corpus. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT ’18, pages 21–27, New Orleans, LA, USA.
    Google ScholarLocate open access versionFindings
  • Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT ’19, pages 4171–4186, Minneapolis, MN, USA.
    Google ScholarLocate open access versionFindings
  • William B. Dolan and Chris Brockett. 2005. Automatically constructing a corpus of sentential paraphrases. In Proceedings of the Third International Workshop on Paraphrasing, IWP ’05, Jeju Island, Korea.
    Google ScholarLocate open access versionFindings
  • Sebastian Dungs, Ahmet Aker, Norbert Fuhr, and Kalina Bontcheva. 20Can rumour stance alone predict veracity? In Proceedings of the 27th International Conference on Computational Linguistics, COLING ’18, pages 3360–3370, Santa Fe, NM, USA.
    Google ScholarLocate open access versionFindings
  • Tamer Elsayed, Preslav Nakov, Alberto BarronCedeno, Maram Hasanain, Reem Suwaileh, Pepa Atanasova, and Giovanni Da San Martino. 20CheckThat! at CLEF 2019: Automatic identification and verification of claims. In Proceedings of the 41st European Conference on Information Retrieval, ECIR ’19, pages 309–315, Cologne, Germany.
    Google ScholarLocate open access versionFindings
  • Mohamed H. Gad-Elrab, Daria Stepanova, Jacopo Urbani, and Gerhard Weikum. 2019a. ExFaKT: A framework for explaining facts over knowledge graphs and text. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, WSDM ’19, pages 87–95, Melbourne, Australia.
    Google ScholarLocate open access versionFindings
  • Mohamed H. Gad-Elrab, Daria Stepanova, Jacopo Urbani, and Gerhard Weikum. 2019b. Tracy: Tracing facts over knowledge graphs and text. In The World Wide Web Conference, WWW ’19, pages 3516–3520, San Francisco, CA, USA.
    Google ScholarLocate open access versionFindings
  • Pepa Gencheva, Preslav Nakov, Lluıs Marquez, Alberto Barron-Cedeno, and Ivan Koychev. 2017. A context-aware approach for detecting worthchecking claims in political debates. In Proceedings of the International Conference on Recent Advances in Natural Language Processing, RANLP ’17, pages 267–276, Varna, Bulgaria.
    Google ScholarLocate open access versionFindings
  • Genevieve Gorrell, Elena Kochkina, Maria Liakata, Ahmet Aker, Arkaitz Zubiaga, Kalina Bontcheva, and Leon Derczynski. 2019. SemEval-2019 task 7: RumourEval, determining rumour veracity and support for rumours. In Proceedings of the 13th International Workshop on Semantic Evaluation, SemEval ’19, pages 845–854, Minneapolis, MN, USA.
    Google ScholarLocate open access versionFindings
  • Naeemul Hassan, Chengkai Li, and Mark Tremayne. 2015. Detecting check-worthy factual claims in presidential debates. In Proceedings of the 24th ACM International Conference on Information and Knowledge Management, CIKM ’15, pages 1835– 1838, Melbourne, Australia.
    Google ScholarLocate open access versionFindings
  • Naeemul Hassan, Gensheng Zhang, Fatma Arslan, Josue Caraballo, Damian Jimenez, Siddhant Gawsane, Shohedul Hasan, Minumol Joseph, Aaditya Kulkarni, Anil Kumar Nayak, Vikas Sable, Chengkai Li, and Mark Tremayne. 2017. ClaimBuster: The firstever end-to-end fact-checking system. Proc. VLDB Endow., 10(12):1945–1948.
    Google ScholarLocate open access versionFindings
  • Viet-Phi Huynh and Paolo Papotti. 2019. A benchmark for fact checking algorithms built on knowledge bases. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM ’19, pages 689–698, Beijing, China.
    Google ScholarLocate open access versionFindings
  • Georgi Karadzhov, Preslav Nakov, Lluıs Marquez, Alberto Barron-Cedeno, and Ivan Koychev. 2017. Fully automated fact checking using external sources. In Proceedings of the Conference on Recent Advances in Natural Language Processing, RANLP ’17, pages 344–353, Varna, Bulgaria.
    Google ScholarLocate open access versionFindings
  • Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv:1907.11692.
    Findings
  • Jing Ma, Wei Gao, Prasenjit Mitra, Sejeong Kwon, Bernard J. Jansen, Kam-Fai Wong, and Meeyoung Cha. 2016. Detecting rumors from microblogs with recurrent neural networks. In Proceedings of the 25th International Joint Conference on Artificial Intelligence, IJCAI ’16, pages 3818–3824, New York, NY, USA.
    Google ScholarLocate open access versionFindings
  • Jing Ma, Wei Gao, and Kam-Fai Wong. 2019. Detect rumors on Twitter by promoting information campaigns with generative adversarial learning. In Proceedings of the World Wide Web Conference, WWW ’19, pages 3049–3055, San Francisco, CA, USA.
    Google ScholarLocate open access versionFindings
  • Tsvetomila Mihaylova, Preslav Nakov, Lluıs Marquez, Alberto Barron-Cedeno, Mitra Mohtarami, Georgi Karadjov, and James Glass. 2018. Fact checking in community forums. In Proceedings of the ThirtySecond AAAI Conference on Artificial Intelligence, AAAI ’18, pages 879–886, New Orleans, LA, USA.
    Google ScholarLocate open access versionFindings
  • Sebastiao Miranda, David Nogueira, Afonso Mendes, Andreas Vlachos, Andrew Secker, Rebecca Garrett, Jeff Mitchel, and Zita Marinho. 2019. Automated fact checking in the news room. In Proceedings of the World Wide Web Conference, WWW ’19, pages 3579–3583, San Francisco, CA, USA.
    Google ScholarLocate open access versionFindings
  • Subhabrata Mukherjee and Gerhard Weikum. 2015. Leveraging joint interactions for credibility analysis in news communities. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, CIKM ’15, pages 353–362, Melbourne, Australia.
    Google ScholarLocate open access versionFindings
  • Moin Nadeem, Wei Fang, Brian Xu, Mitra Mohtarami, and James Glass. 2019. FAKTA: An automatic endto-end fact checking system. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, NAACL-HLT ’19, pages 78–83, Minneapolis, MN, USA.
    Google ScholarLocate open access versionFindings
  • Yixin Nie, Haonan Chen, and Mohit Bansal. 2019. Combining fact extraction and verification with neural semantic matching networks. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence, AAAI ’19, pages 6859–6866, Honolulu, HI, USA.
    Google ScholarLocate open access versionFindings
  • Rodrigo Nogueira and Kyunghyun Cho. 2019. Passage re-ranking with BERT. arXiv:1901.04085.
    Findings
  • Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, and Jimmy Lin. 2019. Multi-stage document ranking with BERT. arXiv:1910.14424.
    Findings
  • Ayush Patwari, Dan Goldwasser, and Saurabh Bagchi. 2017. TATHYA: A multi-classifier system for detecting check-worthy statements in political debates. In Proceedings of the 2017 ACM Conference on Information and Knowledge Management, CIKM ’17, pages 2259–2262, Singapore.
    Google ScholarLocate open access versionFindings
  • Fabio Petroni, Tim Rocktaschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLPIJCNLP ’19, pages 2463–2473, Hong Kong, China.
    Google ScholarLocate open access versionFindings
  • Kashyap Popat, Subhabrata Mukherjee, Jannik Strotgen, and Gerhard Weikum. 2017. Where the truth lies: Explaining the credibility of emerging claims on the web and social media. In Proceedings of the 26th International Conference on World Wide Web Companion, WWW ’17 Companion, pages 1003–1012, Perth, Australia.
    Google ScholarLocate open access versionFindings
  • Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. 2017. Truth of varying shades: Analyzing language in fake news and political fact-checking. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP ’17, pages 2931–2937, Copenhagen, Denmark.
    Google ScholarLocate open access versionFindings
  • Nils Reimers and Iryna Gurevych. 2019. SentenceBERT: Sentence embeddings using Siamese BERTnetworks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP ’19, pages 3980–3990, Hong Kong, China.
    Google ScholarLocate open access versionFindings
  • Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr., 3(4):333–389.
    Google ScholarLocate open access versionFindings
  • Prashant Shiadralkar, Alessandro Flammini, Filippo Menczer, and Giovanni Luca Ciampaglia. 2017. Finding streams in knowledge graphs to support fact checking. In Proceedings of the 2017 IEEE International Conference on Data Mining, ICDM ’17, pages 859–864, New Orleans, LA, USA.
    Google ScholarLocate open access versionFindings
  • 2019. ClaimsKG: A knowledge graph of factchecked claims. In Proceedings of the 18th International Semantic Web Conference, ISWC ’19, pages 309–324, Auckland, New Zealand.
    Google ScholarLocate open access versionFindings
  • James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT ’18, pages 809–819, New Orleans, LA, USA.
    Google ScholarLocate open access versionFindings
  • Slavena Vasileva, Pepa Atanasova, Lluıs Marquez, Alberto Barron-Cedeno, and Preslav Nakov. 2019. It takes nine to smell a rat: Neural multi-task learning for check-worthiness prediction. In Proceedings of the International Conference on Recent Advances in Natural Language Processing, RANLP ’19, pages 1229–1239, Varna, Bulgaria.
    Google ScholarLocate open access versionFindings
  • Soroush Vosoughi, Deb Roy, and Sinan Aral. 2018. The spread of true and false news online. Science, 359(6380):1146–1151.
    Google ScholarLocate open access versionFindings
  • Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–355, Brussels, Belgium.
    Google ScholarLocate open access versionFindings
  • William Yang Wang. 2017. “Liar, liar pants on fire”: A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL ’17, pages 422–426, Vancouver, Canada.
    Google ScholarLocate open access versionFindings
  • Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT ’18, pages 1112–1122, New Orleans, LA, USA.
    Google ScholarLocate open access versionFindings
  • Wei Yang, Haotian Zhang, and Jimmy Lin. 2019. Simple applications of BERT for ad hoc document retrieval. arXiv:1903.10972.
    Findings
  • Tauhid Zaman, Emily B. Fox, and Eric T. Bradlow. 2014. A Bayesian approach for predicting the popularity of tweets. Ann. Appl. Stat., 8(3):1583–1611.
    Google ScholarLocate open access versionFindings
  • Dimitrina Zlatkova, Preslav Nakov, and Ivan Koychev. 2019. Fact-checking meets fauxtography: Verifying claims about images. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, EMNLP ’19, pages 2099–2108, Hong Kong, China.
    Google ScholarLocate open access versionFindings
Full Text
Your rating :
0

 

Tags
Comments