Evaluating the Performance of ChatGPT for Spam Email Detection
CoRR(2024)
摘要
Email continues to be a pivotal and extensively utilized communication medium
within professional and commercial domains. Nonetheless, the prevalence of spam
emails poses a significant challenge for users, disrupting their daily routines
and diminishing productivity. Consequently, accurately identifying and
filtering spam based on content has become crucial for cybersecurity. Recent
advancements in natural language processing, particularly with large language
models like ChatGPT, have shown remarkable performance in tasks such as
question answering and text generation. However, its potential in spam
identification remains underexplored. To fill in the gap, this study attempts
to evaluate ChatGPT's capabilities for spam identification in both English and
Chinese email datasets. We employ ChatGPT for spam email detection using
in-context learning, which requires a prompt instruction and a few
demonstrations. We also investigate how the training example size affects the
performance of ChatGPT. For comparison, we also implement five popular
benchmark methods, including naive Bayes, support vector machines (SVM),
logistic regression (LR), feedforward dense neural networks (DNN), and BERT
classifiers. Though extensive experiments, the performance of ChatGPT is
significantly worse than deep supervised learning methods in the large English
dataset, while it presents superior performance on the low-resourced Chinese
dataset, even outperforming BERT in this case.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要