Generating Diverse Arguments Based on Theme-Level Preference Optimization

Argument generation is a core research task in social media computing and can facilitate multiple securityrelated applications. It aims to automatically generate persuasive and coherent arguments that express supportive or opposite stance towards a given claim. Previous research has mainly emphasized the issues such as aspect control and factual accuracy, largely overlooking the diversity issue in argument generation. Generating diverse arguments is essential to helping users articulate a wide spectrum of personal opinions. With the rapid development of Large Language Models (LLMs), computational argumentation has emerged as an important means to promoting positive online interactions and fostering active community communications among participants. However, it also brings the risk of producing harmful contents in the meanwhile. Thus, generating diverse arguments under ethical considerations is another important issue. In this paper, we propose a Diverse Argument Generation (DAG) framework to address these issues. Our DAG integrates safety-aware supervised fine-tuning and diversity-driven data construction with theme-level preference optimization, which encourages claim-stance relevant and ethics-aware argument generation with thematic and semantic diversity. Specifically, DAG first fine-tunes a model for argument generation and uses the fine-tuned model to generate arguments, both aligning with ethical constraints. It then constructs paired preference data guided with thematic relevance and semantic diversity measures via theme modeling, which are finally utilized by themelevel preference optimization for enhancing diverse argument generation. We conduct experiments to verify the effectiveness of our framework and the results show that DAG achieves better performances in generating diverse and controllable arguments compared to baselines, while maintaining comparable performance in text quality.