AI帮你理解科学

AI 生成解读视频

AI抽取解析论文重点内容自动生成视频


pub
生成解读视频

AI 溯源

AI解析本论文相关学术脉络


Master Reading Tree
生成 溯源树

AI 精读

AI抽取本论文的概要总结


微博一下
We project predictions on comparable data in Bengali, Hindi, and Spanish and we report results of 0.8415 F1 macro for Bengali, 0.8568 F1 macro for Hindi, and 0.7513 F1 macro for Spanish

Multilingual Offensive Language Identification with Cross lingual Embeddings

EMNLP 2020, pp.5838-5844, (2020)

被引用0|浏览291
下载 PDF 全文
引用
微博一下

摘要

Offensive content is pervasive in social media and a reason for concern to companies and government organizations. Several studies have been recently published investigating methods to detect the various forms of such content (e.g. hate speech, cyberbulling, and cyberaggression). The clear majority of these studies deal with English parti...更多

代码

数据

0
简介
  • Offensive posts on social media result in a number of undesired consequences to users.
  • To the best of the knowledge, state-of-the-art cross-lingual contextual embeddings such as XLMR (Conneau et al, 2019) have not yet been applied to offensive language identification.
  • To address this gap, the authors evaluate the performance of cross-lingual contextual embeddings and transfer learning (TL) methods in projecting predictions from English to other languages.
  • The authors take advantage of existing English data to project predictions in three other languages: Bengali, Hindi, and Spanish
重点内容
  • Offensive posts on social media result in a number of undesired consequences to users
  • When we adopt XLM-R for multilingual offensive language identification, we perform transfer learning in two different ways
  • We save the weights of the XLM-R model as well as the softmax layer. We use these saved weights from English to initialise the weights for a new language. To explore this transfer learning aspect we experimented on Hindi language which was released for HASOC 2019 shared task (Mandl et al, 2019) and on Spanish data released for Hateval 2019 (Basile et al, 2019)
  • We have showed that XLM-R with transfer learning outperforms all of the other methods we tested as well as the best results obtained by participants of the three competitions
  • The results obtained by our models confirm that offensive language identification dataset (OLID)’s general hierarchical annotation model encompasses multiple types of offensive content such as aggression, included in the Bengali dataset, and hate speech included in the Hindi and Spanish datasets, allowing us to model different subtasks jointly using the methods described in this paper
  • This opens exciting new avenues for future research considering the multitude of phenomena, annotation schemes and guidelines used in offensive language datasets
方法
  • Transformer models have been used successfully for various NLP tasks (Devlin et al, 2019).
  • There were several multilingual models like BERTm (Devlin et al, 2019) there was much speculations about its ability to represent all the languages (Pires et al, 2019) and BERT-m model showed some cross-lingual characteristics it has not been trained on crosslingual data (Karthikeyan et al, 2020)
  • The motivation behind this methodology was the recently released cross-lingual transformer models - XLM-R (Conneau et al, 2019) which has been trained on 104 languages.
  • This process is known as transfer learning and is illustrated in Figure 1
结果
  • Inter-language transfer learning The authors first trained the XLM-R classification model on first level of English offensive language identification dataset (OLID) (Zampieri et al, 2019a).
  • The authors did not use the weights of the last softmax layer since the authors wanted to test this strategy on data that has a different number of offensive classes to predict
  • The authors explored this transfer learning aspect with Bengali dataset released with TRAC - 2 shared task (Kumar et al, 2020).
  • As described in the Section 3 the classifier should make a 3-way classification in between ‘Overtly Aggressive’, ‘Covertly Aggressive’ and ‘Non Aggressive’ text data
结论
  • This paper is the first study to apply cross-lingual contextual word embeddings in offensive language identification projecting predictions from English to other languages using benchmarked datasets from shared tasks on Bengali (Kumar et al, 2020), Hindi (Mandl et al, 2019), and Spanish (Basile et al, 2019).
  • The authors would like to further evaluate the models using SOLID, a novel large English dataset with over 9 million tweets (Rosenthal et al, 2020), along with datasets in four other languages (Arabic, Danish, Greek, and Turkish) that were made available for the second edition of OffensEval (Zampieri et al, 2020)
  • These datasets were collected using the same methodology and were annotated according to OLID’s guidelines.
  • The authors would like to apply the models to languages with even less resources available to help coping with the problem of offensive language in social media
表格
  • Table1: Instances (Inst.), source (S) and labels in all datasets. F stands for Facebook and T for Twitter
  • Table2: Results ordered by macro (M) F1 for Bengali and weighted (W) F1 for Hindi and Spanish
Download tables as Excel
相关工作
研究对象与分析
datasets: 4
We acquired datasets in English and three other languages: Bengali, Hindi, and Spanish (listed in Table 1). The four datasets have been used in shared tasks in 2019 and 2020 allowing us to compare the performance of our methods to other approaches. As our English dataset, we chose the Offensive Language Identification Dataset (OLID) (Zampieri et al, 2019a), used in the SemEval-2019 Task 6 (OffensEval) (Zampieri et al, 2019b)

datasets: 3
We chose OLID due to the flexibility provided by its hierarchical annotation model that considers multiple types of offensive content in a single taxonomy (e.g. targeted insults to a group are often hate speech whereas targeted insults to an individual are often cyberbulling). This allows us to map OLID level A (offensive vs. non-offensive) to labels in the other three datasets. OLID’s annotation model is intended to serve as a general-purpose model for multiple abusive language detection subtasks (Waseem et al, 2017)

datasets: 3
The Bengali dataset (Bhattacharya et al, 2020) was used in the TRAC-2 shared task (Kumar et al, 2020) on aggression identification. It is different than the other three datasets in terms of domain (Facebook instead of Twitter) and set of labels (three classes instead of binary), allowing us to compare the performance of cross-lingual embeddings on off-domain data and off-task data. Lang

引用论文
  • Segun Taofeek Aroyehun and Alexander Gelbukh. 2018. Aggression detection in social media: Using deep neural networks, data augmentation, and pseudo labeling. In Proceedings of TRAC.
    Google ScholarLocate open access versionFindings
  • Rienke Bannink, Suzanne Broeren, Petra M van de Looij-Jansen, Frouwkje G de Waart, and Hein Raat. 2014. Cyber and Traditional Bullying Victimization as a Risk Factor for Mental Health Problems and Suicidal Ideation in Adolescents. PloS one, 9(4).
    Google ScholarLocate open access versionFindings
  • Md Abul Bashar and Richi Nayak. 2019. QutNocturnal@ HASOC’19: CNN for hate speech and offensive content identification in Hindi language. In Proceedings of FIRE.
    Google ScholarLocate open access versionFindings
  • Valerio Basile, Cristina Bosco, Elisabetta Fersini, Debora Nozza, Viviana Patti, Francisco Manuel Rangel Pardo, Paolo Rosso, and Manuela Sanguinetti. 2019. SemEval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter. In Proceedings of SemEval.
    Google ScholarLocate open access versionFindings
  • Shiladitya Bhattacharya, Siddharth Singh, Ritesh Kumar, Akanksha Bansal, Akash Bhagat, Yogesh Dawer, Bornini Lahiri, and Atul Kr. Ojha. 2020. Developing a multilingual annotated corpus of misogyny and aggression. In Proceedings of TRAC.
    Google ScholarLocate open access versionFindings
  • Rina A Bonanno and Shelley Hymel. 2013. Cyber bullying and internalizing difficulties: Above and beyond the impact of traditional forms of bullying. Journal of youth and adolescence, 42(5):685–697.
    Google ScholarLocate open access versionFindings
  • Cagrı Coltekin. 2020. A Corpus of Turkish Offensive Language on Social Media. In Proceedings of LREC.
    Google ScholarLocate open access versionFindings
  • Patricia Chiril, Farah Benamara Zitoune, Veronique Moriceau, Marlene Coulomb-Gully, and Abhishek Kumar. 2019. Multilingual and multitarget hate speech detection in tweets. In Proceedings of TALN.
    Google ScholarLocate open access versionFindings
  • Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzman, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 201Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116.
    Findings
  • Maral Dadvar, Dolf Trieschnigg, Roeland Ordelman, and Franciska de Jong. 2013. Improving Dyberbullying Detection with User Context. In Proceedings of ECIR.
    Google ScholarLocate open access versionFindings
  • Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL.
    Google ScholarLocate open access versionFindings
  • Darja Fiser, Tomaz Erjavec, and Nikola Ljubesic. 2017. Legal Framework, Dataset and Annotation Schema for Socially Unacceptable On-line Discourse Practices in Slovene. In Proceedings ALW.
    Google ScholarLocate open access versionFindings
  • Paula Fortuna, Joao Rocha da Silva, Leo Wanner, Sergio Nunes, et al. 2019. A Hierarchically-labeled Portuguese Hate Speech Dataset. In Proceedings of ALW.
    Google ScholarLocate open access versionFindings
  • Erfan Ghadery and Marie-Francine Moens. 2020. Liir at semeval-2020 task 12: A cross-lingual augmentation approach for multilingual offensive language identification. arXiv preprint arXiv:2005.03695.
    Findings
  • K Karthikeyan, Zihan Wang, Stephen Mayhew, and Dan Roth. 2020. Cross-lingual ability of multilingual bert: An empirical study. In Proceedings of ICLR.
    Google ScholarLocate open access versionFindings
  • Ritesh Kumar, Atul Kr Ojha, Shervin Malmasi, and Marcos Zampieri. 2018. Benchmarking aggression identification in social media. In Proceedings of TRAC.
    Google ScholarLocate open access versionFindings
  • Ritesh Kumar, Atul Kr. Ojha, Shervin Malmasi, and Marcos Zampieri. 2020. Evaluating Aggression Identification in Social Media. In Proceedings of TRAC.
    Google ScholarLocate open access versionFindings
  • Ping Liu, Wen Li, and Liang Zou. 2019. NULI at SemEval-2019 task 6: Transfer learning for offensive language detection using bidirectional transformers. In Proceedings of SemEval.
    Google ScholarLocate open access versionFindings
  • Prasenjit Majumder, Thomas Mandl, et al. 2018. Filtering Aggression from the Multilingual Social Media Feed. In Proceedings TRAC.
    Google ScholarLocate open access versionFindings
  • Shervin Malmasi and Marcos Zampieri. 2017. Detecting Hate Speech in Social Media. In Proceedings of RANLP.
    Google ScholarLocate open access versionFindings
  • Shervin Malmasi and Marcos Zampieri. 2018. Challenges in Discriminating Profanity from Hate Speech. Journal of Experimental & Theoretical Artificial Intelligence, 30:1 – 16.
    Google ScholarLocate open access versionFindings
  • Thomas Mandl, Sandip Modha, Prasenjit Majumder, Daksh Patel, Mohana Dave, Chintak Mandlia, and Aditya Patel. 2019. Overview of the hasoc track at fire 2019: Hate speech and offensive content identification in indo-european languages. In Proceedings of FIRE.
    Google ScholarLocate open access versionFindings
  • Hamdy Mubarak, Darwish Kareem, and Magdy Walid. 2017. Abusive language detection on Arabic social media. In Proceedings of ALW.
    Google ScholarLocate open access versionFindings
  • Hamdy Mubarak, Ammar Rashed, Kareem Darwish, Younes Samih, and Ahmed Abdelali. 2020. Arabic offensive language on twitter: Analysis and experiments. arXiv preprint arXiv:2004.02192.
    Findings
  • Endang Wahyu Pamungkas and Viviana Patti. 2019. Cross-domain and cross-lingual abusive language detection: A hybrid approach with deep learning and a multilingual lexicon. In Proceedings ACL:SRW.
    Google ScholarLocate open access versionFindings
  • Juan Manuel Perez and Franco M Luque. 2019. Atalaya at semeval 2019 task 5: Robust embeddings for tweet classification. In Proceedings of SemEval.
    Google ScholarLocate open access versionFindings
  • Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep Contextualized Word Representations. In Proceedings of NAACL.
    Google ScholarLocate open access versionFindings
  • Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. How multilingual is multilingual BERT? In Proceedings of ACL.
    Google ScholarLocate open access versionFindings
  • Zeses Pitenis, Marcos Zampieri, and Tharindu Ranasinghe. 2020. Offensive Language Identification in Greek. In Proceedings of LREC.
    Google ScholarLocate open access versionFindings
  • Fabio Poletto, Marco Stranisci, Manuela Sanguinetti, Viviana Patti, and Cristina Bosco. 2017. Hate Speech Annotation: Analysis of an Italian Twitter Corpus. In Proceedings of CLiC-it.
    Google ScholarLocate open access versionFindings
  • Tharindu Ranasinghe, Marcos Zampieri, and Hansi Hettiarachchi. 2019. Brums at hasoc 2019: Deep learning models for multilingual hate speech and offensive language identification. In Proceedings of FIRE.
    Google ScholarLocate open access versionFindings
  • Michael Ridenhour, Arunkumar Bagavathi, Elaheh Raisi, and Siddharth Krishnan. 2020. Detecting Online Hate Speech: Approaches Using Weak Supervision and Network Embedding Models. arXiv preprint arXiv:2007.12724.
    Findings
  • Julian Risch and Ralf Krestel. 2020. Bagging bert models for robust aggression identification. In Proceedings of TRAC.
    Google ScholarLocate open access versionFindings
  • Hugo Rosa, N Pereira, Ricardo Ribeiro, Paula Costa Ferreira, Joao Paulo Carvalho, S Oliveira, Luısa Coheur, Paula Paulino, AM Veiga Simao, and Isabel Trancoso. 2019. Automatic cyberbullying detection: A systematic review. Computers in Human Behavior, 93:333–345.
    Google ScholarLocate open access versionFindings
  • Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Marcos Zampieri, and Preslav Nakov. 2020. A Large-Scale Weakly Supervised Dataset for Offensive Language Identification. In arXiv preprint arXiv:2004.14454.
    Findings
  • Gudbjartur Ingi Sigurbergsson and Leon Derczynski. 2020. Offensive Language and Hate Speech Detection for Danish. In Proceedings of LREC.
    Google ScholarLocate open access versionFindings
  • Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2019. How to fine-tune bert for text classification? In Chinese Computational Linguistics, pages 194– 206.
    Google ScholarLocate open access versionFindings
  • Stephan Tulkens, Lisa Hilte, Elise Lodewyckx, Ben Verhoeven, and Walter Daelemans. 2016. A Dictionary-based Approach to Racism Detection in Dutch Social Media. In Proceedings of TA-COS.
    Google ScholarLocate open access versionFindings
  • Luis Enrique Argota Vega, Jorge Carlos ReyesMagana, Helena Gomez-Adorno, and Gemma BelEnguix. 2019. Mineriaunam at semeval-2019 task 5: Detecting hate speech in twitter using multiple features in a combinatorial framework. In Proceedings of SemEval.
    Google ScholarLocate open access versionFindings
  • Zeerak Waseem, Thomas Davidson, Dana Warmsley, and Ingmar Weber. 2017. Understanding Abuse: A Typology of Abusive Language Detection Subtasks. In Proceedings of ALW.
    Google ScholarLocate open access versionFindings
  • Jun-Ming Xu, Kwang-Sung Jun, Xiaojin Zhu, and Amy Bellmore. 2012. Learning from bullying traces in social media. In Proceedings of NAACL.
    Google ScholarLocate open access versionFindings
  • Mengfan Yao, Charalampos Chelmis, and DaphneyStavroula Zois. 2019. Cyberbullying Ends Here: Towards Robust Detection of Cyberbullying in Social Media. In Proceedings of WWW.
    Google ScholarLocate open access versionFindings
  • Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019a. Predicting the type and target of offensive posts in social media. In Proceedings of NAACL.
    Google ScholarLocate open access versionFindings
  • Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019b. SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval). In Proceedings of SemEval.
    Google ScholarLocate open access versionFindings
  • Marcos Zampieri, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis, and Cagrı Coltekin. 2020. Semeval-2020 task 12: Multilingual offensive language identification in social media (offenseval 2020). Proceedings of SemEval.
    Google ScholarLocate open access versionFindings
作者
Tharindu Ranasinghe
Tharindu Ranasinghe
您的评分 :
0

 

标签
评论
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科