This research studies how Technical Support Scams (TSS) are being countered by a uniquely dedicated community of volunteer counter-fraud operatives. Using a careful subject selection strategy, we interviewed 17 individuals who actively engage in TSS scambaiting activities in order to obtain insight into their motivations, the operational methods of the scammers they combat, the undocumented nuances of effective scambaiting action, and the various challenges scambaiters face. In our analysis, we find a community rich not only with insight into offenders, but with technical and operational expertise that is often lacking in research efforts targeting these same populations. At the same time, we find key areas where the community could be better supported and enabled. We discuss the implications of our findings for both future research and community protection strategies.
This study evaluates scalable, unsupervised methods for detecting malicious online content by benchmarking transformer-based architectures across diverse linguistic contexts. We demonstrate that the semantic resolution of the embedding layer is the critical determinant of performance. Our results identify that large language models coupled with manifold learning achieve superior anomaly separation in knowledge-intensive domains, significantly outperforming traditional BERT-based pipelines. Additionally, we reveal a topological dichotomy in detection strategies: Contrastive autoencoders offer robust stability in structured environments, whereas few-shot deviation learning (FATE) is essential for high-entropy, dynamic contexts such as politics and sports. These findings propose a shift toward context-aware architectures capable of adapting to the complex semantic landscape of modern web content.
Children’s daily use of the Internet exposes them to various online harms, which constitutes any online material or interactions that negatively impacts a child’s well-being. Parental controls and monitoring are often insufficient to mitigate these risks, and children without parental support are especially vulnerable. Consequently, there is motivation for developing automatic systems capable of detecting these risks. To create effective detection mechanisms requires suitable datasets, both to train machine learning systems and to evaluate their real-world effectiveness. To that end, this article employs a narrative review methodology to examine the availability and suitability of resources across four types of online risks posed to children: online grooming, cyberbullying, mental health, and extremism, radicalisation, and hate speech. Importantly, this article highlights a stark lack of data pertaining to children for these detection tasks, identifying a clear research gap. Additionally, we highlight important considerations for determining whether a resource is suitable for the detection task at hand and outline the limitations of currently available resources. By highlighting the deficiencies in both the availability and suitability of resources, our article calls for further research and development to bridge these gaps, offering actionable steps to advance the field and to ultimately ensure children’s safety online.
Amid major global conflicts, groups perpetrate Distributed Denial-of-Service (DDoS) attacks to take direct action and challenge traditional political systems. From 2023 to 2024, Anonymous Sudan launched numerous successful and high-profile DDoS attacks against nations and international businesses, resulting in millions of dollars in damages, drawing global headlines, and culminated in charges not just for criminal hacking, but also for seeking to cause injury or death [1]. DDoS attacks are often analysed through a cybersecurity lens, to offer security solutions to mitigate risk and limit the impact of attacks. However, this may overlook the sociopolitical reasons why groups feel compelled to take such actions. In response, adopting a collective action approach, this study investigated how Anonymous Sudan framed their psychological and political motivations on their public Telegram channels ($\mathbf{N} \boldsymbol{=} \mathbf{9 6 7}$ posts). Using content analysis, we found the primary focus of the posts was to claim attribution for the attacks and provide evidence for these claims. This messaging allowed the group to imbue the attacks with political meaning and frame themselves as strong and significant. However, the need for these communications also reveals the challenges facing groups reliant on the attribution assessments of others, and thereby offers opportunities for developing strategies to counter dangerous actions.
This paper examines the phenomenon of ‘prevention of prosecution’: the methods, tools, and services that cybercriminals use to escape justice. In a semi-automated investigation spanning ten different underground forums, we classify content by the type of prosecution avoidance strategy and the level of support involved (free advice, guides for purchase, or services marketed to customers), allowing us to characterise the different provision across different offender populations. Our investigation provides insight into the legal concerns of cybercriminals, their priorities amongst the different options for countering law enforcement action, and possible disruption points for law enforcement agencies or other actors seeking to redress the poor rates of prosecution for cybercrime. In particular, we identify a range of free advice of variable quality, a small number of services explicitly marketed with the aim of defeating law enforcement, and signs of unmet demand that could be exploited by careful operations.
The practice of microtargeting in politics, involving tailoring persuasive messages to individuals based on personal vulnerabilities, has raised manipulation concerns. As microtargeting’s persuasive benefits are well-established and its use facilitated by AI tools and personality-inference models, ethical and regulatory concerns are magnified. Here, we explore countering microtargeting effects by creating a warning signal deployed when users encounter personality-tailored political ads. Three studies evaluated the effectiveness of warning “popups” against potential microtargeting by comparing persuasiveness of targeted vs. non-targeted messages with and without popups. Using within subject-designs, Studies 1 (N = 666), 2a (N = 432), and 2b (N = 669) reveal a targeting effect, with targeted ads deemed more persuasive than non-targeted ones. More important, the presence of a warning popup had no meaningful impact on persuasiveness. Overall, across the three studies, personality-targeted ads were significantly more persuasive than non-targeted ones, and this advantage persisted despite warnings. Given the focus on transparency in initiatives like the EU’s AI Act, our finding that warnings have little effect has potential policy implications. Warnings about personality-based microtargeting in political ads fail to reduce their persuasiveness, as shown in three studies. Targeted ads remain significantly more persuasive than non-targeted ones, raising concerns for transparency-focused policies.
Large Language Models (LLMs) offer promising opportunities for automating online extremism detection, yet challenges remain in capturing nuanced rhetoric and complex social dynamics. In this study, we evaluate state-of-the-art LLMs (GPT-4o, LLaMA, Gemini, and Mixtral) using the updated UK government definition of extremism, analysing 500 user examples. GPT-4o notably achieved an accuracy of 80
The convergence of information and operational technology networks has created previously unforeseen security issues. To address these issues, both researchers and practitioners have integrated threat intelligence methods into the security operations of converged networks, with some of the most valuable tools being honeypots that imitate industrial control systems (ICS). However, the development and deployment of such honeypots is a process rich with pitfalls, which can lead to undiagnosed weaknesses in the threat intelligence being gathered. This paper presents a side-channel method of covertly identifying ICS honeypots using the time-to-live (TTL) values of target devices. We show that many ICS honeypots can be readily identified, via minimal interactions, using only basic networking tools. In a study of over 8,000 devices presenting as ICS systems, we detail how our method compares to an existing honeypot detection approach, and outline what our methodology reveals about the current population of live ICS honeypots. In demonstrating our method, this study aims to raise awareness of the viability of the TTL heuristic and the prevalence of its misconfiguration despite its presence in literature.
Most research on online collective action investigates low‐effort, social media‐based actions rather than tactics with highly disruptive potential. To better account for the variety of forms of collective actions that use digital technologies, we conducted an open‐source intelligence search (Study 1a) and an expert consultation survey (Study 1b; N = 21), to create a database containing 31 types of actions. In Study 2, we interviewed activists ( N = 20) and found six key dimensions underlying those actions. In Study 3, participants ( N = 273) rated the actions across the dimensions. Based upon the (dis)similarities of each action's rating across the dimensions, we identified two main types and five subtypes of online collective actions: Ingroup‐assisting actions (collaborative resource generation, ingroup mobilization, and digital picketing) and outgroup‐attacking actions (disruptive clicktivism and technology‐enabled attacks). The results showed that digital collective actions substantively differ from each other based on the six underlying dimensions, from the social psychological function, to the skill required, to the groups being targeted. This work offers a multi‐dimensional explanation for the variations across the domain of online activism and offers a way forward for future collective action work to explore psychological motivations underlying choices across action type.
As the evolution of cybercrime has criminals expanding into ever-stranger frontiers, we explore Roblox as an avenue for crime, revealing new methods for old crimes and how the structure of an innocuous children’s game can enable or encourage new crimes.
The practice of microtargeting in politics, involving tailoring persuasive messages to individuals based on personal vulnerabilities, has raised manipulation concerns. As microtargeting's persuasive benefits are well-established and its use facilitated by AI tools and personality-inference models, ethical and regulatory concerns arise. Here, we explore countering microtargeting effects by creating a warning signal deployed when users encounter personality-tailored political ads. Two studies evaluated the effectiveness of warning "popups'' against potential microtargeting by comparing persuasiveness of targeted vs. non-targeted messages with and without popups. Using within subject-designs, Studies 1 (N = 666) and 2 (N = 432) reveal a targeting effect, with targeted ads deemed more persuasive than non-targeted ones. More importantly, the presence of a warning popup did not significantly impact persuasiveness. Overall, across the two studies, personality-targeted ads were significantly more persuasive than non-targeted ones, and this advantage persisted despite warnings. Given the focus on transparency in initiatives like the EU's AI Act, the lack of a significant transparency measure effect is potentially concerning.
We ask whether state-of-the-art large language models can provide a viable alternative to human annotators for detecting and explaining behavioural influence online. Working with a large corpus of online interactions retrieved from the social media platform Mastodon, we cross-examine a dataset containing 11,000 LLM influence labels and explanations across nine state-of-the-art large language models from 312 scenarios. We use a range of resolution categories and four stages of shot prompting to further measure the importance of context to language model performance. We also consider the impact of model architecture, and how social media content and features from the explanation impact model labelling accuracy. Our experiment shows that whilst most large language models struggle to identify the correct framing of influence from an interaction, at lower label resolutions, models like Flan and GPT-4 Turbo perform with an accuracy of 70%-80%, demonstrating encouraging potential for future social influence identification and explanation, and contributing to our understanding of the general social reasoning capabilities of large language models.
Automatic scam-baiting is an online fraud countermeasure that involves automated systems responding to online fraudsters in order to waste their time and deplete their resources, diverting attackers away from real potential victims. Previous work has demonstrated that text generation systems are capable of engaging with attackers as automatic scam-baiters, but the fluency and coherence of generated text may be a limit to the effectiveness of such systems. In this paper, we report on the results of a month-long experiment comparing the effectiveness of two ChatGPT-based automatic scam-baiters to a control measure. Within our results, with engagement from over 250 real email fraudsters, we find that ChatGPT-based scam-baiters show a marked increase in scammer response rate and conversation length relative to the control measure, outperforming previous approaches. We discuss the implications of these results and practical considerations for wider deployment of automatic scam-baiting.
Powerful generative Large Language Models (LLMs) are becoming popular tools amongst the general public as question-answering systems, and are being utilised by vulnerable groups such as children. With children increasingly interacting with these tools, it is imperative for researchers to scrutinise the safety of LLMs, especially for applications that could lead to serious outcomes, such as online child safety queries. In this paper, the efficacy of LLMs for online grooming prevention is explored both for identifying and avoiding grooming through advice generation, and the impact of prompt design on model performance is investigated by varying the provided context and prompt specificity. In results reflecting over 6,000 LLM interactions, we find that no models were clearly appropriate for online grooming prevention, with an observed lack of consistency in behaviours, and potential for harmful answer generation, especially from open-source models. We outline where and how models fall short, providing suggestions for improvement, and identify prompt designs that heavily altered model performance in troubling ways, with findings that can be used to inform best practice usage guides.
The increasing availability of microtargeted advertising and the accessibility of generative AI tools, such as ChatGPT, have raised concerns about the potential misuse of large language models (LLMs) in scaling microtargeting efforts for political purposes. Recent technological advancements, involving generative AI and personality inference from consumed text, can potentially create a highly scalable "manipulation machine'" that targets individuals based on their unique vulnerabilities without requiring human input. This paper presents four studies examining the effectiveness of this putative "manipulation machine'". The results demonstrate that personalized political ads tailored to individuals' personalities are more effective than non-personalized ads (Studies 1a and 1b). Additionally, we showcase the feasibility of automatically generating and validating these personalized ads on a large scale (Studies 2a and 2b). These findings highlight the potential risks of utilizing AI and microtargeting to craft political messages that resonate with individuals based on their personality traits. This should be an area of concern to ethicists and policy makers.
With the development and growing use of social media platforms in the last two decades, platform architectures have driven how we notice, consume and share information. Whilst centralised social networks and the use of recommendation algorithms are a prominently used architecture, in recent years an alternative and novel framework has emerged aiming to offer users a non-commercial decentralised platform to distribute content. Run by users of the platform, Mastodon offers many of the benefits of traditional centralised approaches, however, with the absence of recommendation algorithms there is risk that these architectures could instead promote echo-chambers and the growth of disinformation. With this in mind, we collect a new large Mastodon dataset, consisting of three million connections between over a hundred thousand users. Modelling content using 68 conversational features, and measuring influence using twelve different metrics, we analyse the most common topics being discussed between influential users, the conversational features present in influential content, and the relationships between influence measurements. Our analysis finds a strong correlation between influence and negative traits at every network resolution, with positive and neutral traits in some cases being negatively correlated with influence. Our analysis also shows that influential users have a strong relationship with social/political commentary.
Email-based fraud is a lucrative market for cybercriminals to scam a wide range of potential victims. Yet there is a sometimes conflicted literature on who these victims are, complicated by low and possibly confounded reporting rates. We make use of an experimental automated scam-baiting platform to test hypotheses about the characteristics online fraudsters find more attractive, gathering behavioural evidence directly from the fraudsters themselves ( n = 296). In our comparison of four instrumented ‘personalities’ designed based on traits highlighted in the literature and in a small public perception survey, we find that a script adopting the personality of an elderly woman attracts significantly more engagement from scammers than our control measure. We discuss our approach and the possible interpretations and implications of our findings.
The internet has revolutionised retail sales, with online shopping a common practice globally. While convenient, offenders have also embraced the opportunity to target potential victims and their shopping carts. Online shopping fraud occurs when offenders represent themselves as legitimate online sellers to gain sales from unsuspecting victims, both by impersonating genuine retailers and creating fictional retailers with non-existent products. The current article explores the victimisation and near misses of consumers to online shopping fraud. Based on survey responses of 1011 Australians, the article examines the online shopping activities of individuals as well as any victimisation or near miss experiences. The results indicate a high level of victimisation and near misses across this sample. It further examines a range of impacts experienced by these consumers and considers the implications of these results for the retail sector and prevention practices into the future.
Online information sources have a considerable influence on the security of applications developed. Prior research has shown that insecure code snippets and security advice is pervasive in popular information sources. Developer-Centered Security (DCS) as a field has suggested interventions with respect to the usage of such information sources. We argue that such interventions are based only on the study of the demand side, where as interventions to be effective in any production environment needs to be situated on a comprehensive understanding of both demand side & supply side. We study the supply side of a popular source for security knowledge - Security Stack Exchange. Our findings reveal that the manner in which the forum identifies its top answerers is inadequate. We find 424 additional answerers whose engagement and topic interests are similar to the ranked top answerers. We term the collective of these power users irrespective of their reputation as Community Security Champion (CSC)s. The significance of this work lie in equipping the community with the information on where to intervene and how to intervene. Our work can serve as a methodological foundation to study network characteristics which is critical for improved user experience and to keep information networks relevant.
Online shopping has now become very common, with consumers increasingly opting to purchase products on the World Wide Web instead of visiting traditional "bricks and mortar" stores, particularly during the COVID-19 pandemic. This has, however, also provided significant opportunities for offenders to abuse the inherent trust-based nature of online shopping, whereby consumers typically do not see the products physically prior to purchasing them. As such, this article sets out to examine the actions and behaviors that individuals take to prevent online shopping fraud and what, if any, discrepancy exists between prevention messaging and consumer behavior. To accomplish this, the study utilizes secondary survey data (n = 3478 respondents) obtained from a private-sector initiative called ScamAdviser. The results find that many respondents do not use appropriate behaviors to reduce their risk when shopping online and that furthermore there is reason to believe that consumers are not served well by the online safety advice that they are given. The paper argues that there is scope to increase guardianship through better prevention advice being communicated to online shoppers.
Paul Rayson合作论文数School of Computing and Communications, Lancaster University4