Task and Data Complexity in Gamified Text Labelling: when Do Real and AI-Generated Images Help?

This paper examines whether AI-generated images improve user experience and performance in gamified text-labelling tasks. Across two experiments, we investigate how images interact with both task complexity and data ambiguity. In Experiment 1, participants completed either a simple part-of-speech (POS) tagging task or a more complex natural language inference (NLI) task, with or without AI-generated images. Results showed that task complexity strongly influenced accuracy, engagement, enjoyment, and cognitive load, while AI-generated images provided no consistent benefit. Notably, in the POS task, engagement was higher in the absence of images. Experiment 2 focused on data complexity within NLI, comparing AI-generated images, Flickr images, and a no-image condition across low- and high-ambiguity items. Even when images were designed to be semantically aligned with the text, visual context did not improve engagement or accuracy, nor mitigate the effects of ambiguity; accuracy was often highest without images. Together, these findings suggest that in linguistic annotation tasks, performance and experience are driven primarily by textual complexity rather than visual support, challenging assumptions about benefits of adding images to gamified text annotation.