
The proliferation of misinformation demands the development of automated fact-checking systems. Large language models (LLMs), are increasingly being used for academic, legal, and journalistic content writing. This underscores the critical importance of LLMs in distinguishing between factual accuracy and inaccuracy. Hence, understanding the capacities and limitations of LLMs in fact-checking tasks is essential for their usage in information space. While previous research showed the potential of LLMs in fact-checking English news headlines, the extent to which LLMs work well in other languages are mostly unexplored. In this paper, using data from a local fact-checking website, we investigate the performance of close- and open-source LLMs in fact-checking Farsi news headlines. Our results show that in none of the model combinations, the fact-checking accuracy of LLMs exceeds 55
This paper explores the use of crowdsourcing to classify statement types in film reviews to assess their information quality. Employing the Argument Type Identification Procedure which uses the Periodic Table of Arguments to categorize arguments, the study aims to connect statement types to the overall argument strength and information reliability. Focusing on non-expert annotators in a crowdsourcing environment, the research assesses their reliability based on various factors including language proficiency and annotation experience. Results indicate the importance of careful annotator selection and training to achieve high inter-annotator agreement and highlight challenges in crowdsourcing statement classification for information quality assessment.
Conspiracy theories pose a significant societal challenge, particularly online where their spread can be hard to detect. Robust detection models are crucial for effectively identifying these theories. In this study, we investigate incorporating emotional sentiment and moral framing features into a text-based conspiracy detection model. We hypothesize that incorporating these psycho-linguistic elements would enhance the model's performance. Our results reveal significant psycho-linguistic differences between conspiracy and non-conspiracy texts. Conspiracy texts contain higher levels of anger and are framed through the moral lens of cheating, while non-conspiracy texts contain higher levels of joy and are framed through the moral lenses of care and harm. Our model's ability to classify conspiratorial text improves after integrating emotional sentiment and moral framing into the text-based conspiracy detection model. This work demonstrates the potential value of incorporating psycho-linguistic features into text-based models to enhance conspiracy theory detection.
Can warning labels on social media posts reduce the spread of misinformation online? This paper presents the results of an empirical study using ModSimulator, an open-source mock social media research tool, to test the effectiveness of soft moderation interventions aimed at limiting misinformation spread and informing users about post accuracy. Specifically, the study used ModSimulator to create a social media interface that mimics the experience of using Facebook and tested two common soft moderation interventions – a footnote warning label and a blur filter – to examine how users (n = 1500) respond to misinformation labels attached to false claims about the Russia-Ukraine war. Results indicate that both types of interventions decreased engagement with posts featuring false claims in a Facebook-like simulated interface, with neither demonstrating a significantly stronger effect than the other. In addition, the study finds that belief in pro-Kremlin claims and trust in partisan sources increase the likelihood of engagement, while trust in fact-checking organizations and frequent commenting on Facebook lowers it. These findings underscore the importance of not solely relying on soft moderation interventions, as other factors impact users’ decisions to engage with misinformation on social media.
False claims about vaccines can find large audiences online, leading to vaccine hesitancy. The most influential content on social media is often visual, but studies about misinformation largely focus on text instead of images. This study uses newimage analysis capabilities that Facebook and Instagram have made available to understand the spread of visual anti-vaccinationmemes on these platforms. We identified 200 influential memes that contain scepticism or hesitancy towards vaccines and the 15,000 public Facebook accounts on which the memes have been shared. We describe the memes' spread on a large scale by identifying communities of accounts and describing the diffusion pathways of memes between the communities. We develop a novel method of testing whether a meme has spread from one community of accounts to another that works on sequential time series alone. We identify 16 distinct communities of Facebook accounts and categories them based on thematic and regional focus. Anti-vaccination memes originate predominantly from North American Facebook accounts. These accounts often focus on opposing COVID-19 policies or promoting conspiracy theories about elites. Memes from these communities also spread internationally, particularly to Europe, demonstrating their influence beyond North America. The analysis demonstrates that memes receive the most engagement within their initial community. However, their overall reach depends on their ability to spread to other communities. This suggests that the ability of memes to find large audiences is based on their capacity to spread beyond their original contexts and to be used by groups with potentially different agendas.