M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection.

Yuxia Wang,Jonibek Mansurov,Petar Ivanov,Jinyan Su,Artem Shelmanov,Akim Tsvigun,Chenxi Whitehouse,Osama Mohammed Afzal,Tarek Mahmoud, Toru Sasaki,Thomas Arnold,Alham Fikri Aji,Nizar Habash,Iryna Gurevych,Preslav Nakov

Computing Research Repository (CoRR)（2024）

Mohamed bin Zayed Univ Artificial Intelligence | Tech Univ Darmstadt

Cited 29|Views69

Abstract

Large language models (LLMs) have demonstrated remarkable capability togenerate fluent responses to a wide variety of user queries. However, this hasalso raised concerns about the potential misuse of such texts in journalism,education, and academia. In this study, we strive to create automated systemsthat can detect machine-generated texts and pinpoint potential misuse. We firstintroduce a large-scale benchmark M4, which is a multi-generator,multi-domain, and multi-lingual corpus for machine-generated text detection.Through an extensive empirical study of this dataset, we show that it ischallenging for detectors to generalize well on instances from unseen domainsor LLMs. In such cases, detectors tend to misclassify machine-generated text ashuman-written. These results show that the problem is far from solved and thatthere is a lot of room for improvement. We believe that our dataset will enablefuture research towards more robust approaches to this pressing societalproblem. The dataset is available at https://github.com/mbzuai-nlp/M4.

Translated text

Key words

Language Modeling,Detection,Natural Language Processing,Multilingual Neural Machine Translation,Complex Word Identification

Bibtex

AI Read Science

AI Summary

AI Summary is the key point extracted automatically understanding the full text of the paper, including the background, methods, results, conclusions, icons and other key content, so that you can get the outline of the paper at a glance.

Example

Background

Key content

Introduction

Methods

Results

Related work

Fund

Key content

Pretraining has recently greatly promoted the development of natural language processing (NLP)
We show that M6 outperforms the baselines in multimodal downstream tasks, and the large M6 with 10 parameters can reach a better performance
We propose a method called M6 that is able to process information of multiple modalities and perform both single-modal and cross-modal understanding and generation
The model is scaled to large model with 10 billion parameters with sophisticated deployment, and the 10 -parameter M6-large is the largest pretrained model in Chinese
Experimental results show that our proposed M6 outperforms the baseline in a number of downstream tasks concerning both single modality and multiple modalities We will continue the pretraining of extremely large models by increasing data to explore the limit of its performance

Try using models to generate summary,it takes about 60s

Must-Reading Tree

Example

Generate MRT to find the research sequence of this paper