Combining latent profile analysis and programming traces to understand novices’ differences in debugging

It is widely recognized that debugging is challenging for novice programmers and, as such, computing educators and researchers have called for explicit debugging instruction. Debugging requires various knowledge and skills, and different students may show different strengths and weaknesses. An understanding of such individual differences is important as it may guide personalized instruction. The current study investigated individual differences in debugging in an undergraduate introductory computer science course. We extracted variables related to debugging from students’ submission traces to programming problems in the first month of the course. We applied latent profile analysis to these variables and identified three distinctive profiles. Profile A showed higher debugging accuracy and speed. Profile B showed lower debugging performance in runtime and logic errors, while profile C had lower performance in syntactic errors and tended to make large code edit every submission. Students’ gender and self-rated programming ability predicted profile membership. Moreover, profile A got higher scores than the others in the first exam, and this difference persisted in the second and third exam, even controlling for background variables and score on the first exam. We investigated how students transitioned across debugging profiles over the duration of the course. From the beginning to the end of the course, a large part of students stayed in lower performance profiles. Overall, these findings support the call that debugging should be taught at an early stage and suggest that different groups may need different debugging instructions or support.