Improving Online Anti-Phishing Training Using Cognitive Large Language Models

Training the public to recognize phishing emails is challenging because attackers increasingly use Large Language Models (LLMs) to craft more convincing scams. Although previous research suggests that fully LLM-generated phishing emails are easier for humans to identify, recent models like GPT-4 used in tandem with prompt engineering methods could generate phishing emails that are more difficult to detect. Various training techniques have been used to educate end users about the risks of phishing emails, but these methods may not address the challenges posed by LLM-generated phishing emails. To investigate these challenges, we conducted an experiment using multiple different methods that cyber attackers can use to develop phishing emails. We found that the most challenging emails to detect were those written by humans and then edited and stylized using code generated by an LLM. We also created an anti-phishing training platform to study a personalized learning approach using cognitive models. The basic idea, borrowed from tutoring systems, is to use a cognitive model based on Instance-Based Learning (IBL) Theory to trace the student’s actions and predict each email’s classification decision. The training protocol then selects educational examples that maximize the student’s classification improvement. We also used the predictions of the IBL model to prompt the LLM to generate written feedback for the students. Our study shows that our training methods significantly improved the student’s classification decisions. These results are highly relevant to researchers interested in the social impacts of LLM, their misuse, and mitigation strategies and to practitioners interested in novel anti-phishing training techniques.