Detecting Bot on GitHub Leveraging Transformer-based Models: A Preliminary Study.

Bots are prevalent contributors in collaborative software development, necessitating accurate detection techniques. This preliminary study aims to leveraging public datasets and Transformer-based models (i.e., BERT, CodeBERT, RoBERTa, BART, and PLBART) for the bot detection task. Our experimental result reveals that CodeBERT achieves the highest performance, with an impressive accuracy score of 94.1%.