Generating Feedback-Ladders for Logical Errors in Programming using Large Language Models
arxiv(2024)
摘要
In feedback generation for logical errors in programming assignments, large
language model (LLM)-based methods have shown great promise. These methods ask
the LLM to generate feedback given the problem statement and a student's
(buggy) submission. There are several issues with these types of methods.
First, the generated feedback messages are often too direct in revealing the
error in the submission and thus diminish valuable opportunities for the
student to learn. Second, they do not consider the student's learning context,
i.e., their previous submissions, current knowledge, etc. Third, they are not
layered since existing methods use a single, shared prompt for all student
submissions. In this paper, we explore using LLMs to generate a
"feedback-ladder", i.e., multiple levels of feedback for the same
problem-submission pair. We evaluate the quality of the generated
feedback-ladder via a user study with students, educators, and researchers. We
have observed diminishing effectiveness for higher-level feedback and
higher-scoring submissions overall in the study. In practice, our method
enables teachers to select an appropriate level of feedback to show to a
student based on their personal learning context, or in a progressive manner to
go more detailed if a higher-level feedback fails to correct the student's
error.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要