BERT Model-Based Approach for Detecting Racism and Xenophobia on Twitter Data.

The large amount of data generated on social networks makes the task of moderating textual content written by users complex and impossible to do manually. One of the most prominent problems on social networks is racism and xenophobia. Although there are studies of predictive models that make use of natural language processing techniques to detect racist or xenophobic texts, a lack of these has been observed in the Spanish language. In this paper we present a solution based on deep learning models and, more specifically, models based on transfer learning to detect racist and xenophobic messages in Spanish. For this purpose, a dataset obtained from the social network Twitter has been created using data mining techniques and, after a preprocessing, it has been labelled into racist messages and non-racist messages. The trained models are based on BERT and were called BETO and mBERT. Promising results were obtained showing 85.14% accuracy in the best performing model.