Students' consistency in computational modeling and their academic success

In this study, an assessment was designed to measure consistency in how subjects interpreted the effect of programming statements. The assessment consisted of 24 multiple-choice items which tested student interpretation of assignment and equality operators. Answers were analyzed to determine each subject's "Consistency Score," which represents their consistency in this interpretation. The assessment was administered to computer science undergraduates at a public research university in the Northeast USA. The respondent results (n=128) were compared to the students' self-reported department GPA with the goal of determining whether consistency is correlated with student success. We found a positive correlation between a student's Consistency Score and their department GPA, with strong significance. This suggests the use of this instrument as a diagnostic for supporting students. This paper presents the design of the assessment, how the Consistency Score is calculated, and the study results.