HalluTrigger: Triggering Input-Conflicting Hallucinations in Large Language Models with Semantic-Guided Metamorphic Relations

With rapid development of Large Language Models (LLMs), they are increasingly being adopted as knowledge-based intelligent systems to support decision-making across domains such as education, healthcare, business, and government. Alongside these strides, LLMs have also raised concerns about their security and reliability. One such concern is Input-Conflicting Hallucination (ICH), a type of hallucination that conflicts with user input. Since the data annotation in NLP tasks is expensive and labor-intensive, existing ICH attack methods have adopted Metamorphic Testing (MT) to bypass the oracle problem, i.e. check the ICHs without the ground truths. However, these attacks are all black-box methods utilizing a limited set of Metamorphic Relations (MRs) and can be easily defended against by decoder-only LLMs. In response, we propose HalluTrigger, a simple yet effective grey-box method that constructs six semantic-guided MRs to generate attack cases, and propose a diversity-guided MP prioritization method to promote the trigger of ICHs in target LLMs. We evaluate HalluTrigger on four NLP datasets and three popularly used target LLMs. As a result, the designed MRs achieve higher hallucination trigger rates than existing state-of-the-art baselines, and the diversity-guided MP prioritization can efficiently trigger ICHs with less time.