A Study on the Vulnerability of Test Questions against ChatGPT-based Cheating
2023 International Conference on Machine Learning and Applications (ICMLA)(2024)
摘要
ChatGPT is a chatbot that can answer text prompts fairly accurately, even
performing very well on postgraduate-level questions. Many educators have found
that their take-home or remote tests and exams are vulnerable to ChatGPT-based
cheating because students may directly use answers provided by tools like
ChatGPT. In this paper, we try to provide an answer to an important question:
how well ChatGPT can answer test questions and how we can detect whether the
questions of a test can be answered correctly by ChatGPT. We generated
ChatGPT's responses to the MedMCQA dataset, which contains over 10,000 medical
school entrance exam questions. We analyzed the responses and uncovered certain
types of questions ChatGPT answers more inaccurately than others. In addition,
we have created a basic natural language processing model to single out the
most vulnerable questions to ChatGPT in a collection of questions or a sample
exam. Our tool can be used by test-makers to avoid ChatGPT-vulnerable test
questions.
更多查看译文
关键词
machine learning,data analysis,ChatGPT,NLP
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要