Recognizing Implicitly Toxic Content Based on Multiple Attention Mechanisms

ABSTRACTToxic content on social media has posed a significant threat to user experience and societal stability. In contrast to explicitly toxic content, implicitly toxic content, lacking overt toxic words, is challenging to identify through single‐text features. Therefore, this study proposes a multi‐feature fusion algorithm to recognize implicit toxic content. Firstly, we collect various features for each post, including text content, likes, comments, user information and other features. Subsequently, employing a multi‐head attention mechanism, we extract and fuse these features. Then, utilizing ensemble learning algorithms, we identify implicitly toxic content based on the fused features. Finally, we analyze the decision‐making process of the model using an interpretable algorithm and derive the most critical features and factors for identifying implicitly toxic content. The experimental results show that the proposed algorithm outperforms algorithms such as BERT, TextCNN, and XGBoost, demonstrating the advantage of multiple features over a single text feature in recognizing implicitly toxic content. In addition, this study provides insights into the decision‐making process of the model and provides more effective toxic content management strategies for social media platforms.