Moderating Illicit Online Image Promotion for Unsafe User-Generated Content Games Using Large Vision-Language Models
arxiv(2024)
摘要
Online user-generated content games (UGCGs) are increasingly popular among
children and adolescents for social interaction and more creative online
entertainment. However, they pose a heightened risk of exposure to explicit
content, raising growing concerns for the online safety of children and
adolescents. Despite these concerns, few studies have addressed the issue of
illicit image-based promotions of unsafe UGCGs on social media, which can
inadvertently attract young users. This challenge arises from the difficulty of
obtaining comprehensive training data for UGCG images and the unique nature of
these images, which differ from traditional unsafe content. In this work, we
take the first step towards studying the threat of illicit promotions of unsafe
UGCGs. We collect a real-world dataset comprising 2,924 images that display
diverse sexually explicit and violent content used to promote UGCGs by their
game creators. Our in-depth studies reveal a new understanding of this problem
and the urgent need for automatically flagging illicit UGCG promotions. We
additionally create a cutting-edge system, UGCG-Guard, designed to aid social
media platforms in effectively identifying images used for illicit UGCG
promotions. This system leverages recently introduced large vision-language
models (VLMs) and employs a novel conditional prompting strategy for zero-shot
domain adaptation, along with chain-of-thought (CoT) reasoning for contextual
identification. UGCG-Guard achieves outstanding results, with an accuracy rate
of 94
in real-world scenarios.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要