A Closer Look at Metacognitive Scaffolding: Solving Test Cases Before Programming

Prior research has shown that novices face metacognitive difficulties associated with understanding problem statements. Metacognition refers to the processes that an individual uses to plan, monitor, and assess their own understanding and performance and is often described as "thinking about thinking." Metacognitive awareness is a higher-order thinking skill that includes knowing when and how to use particular strategies for problem-solving. However, novice students often lack metacognitive skills. This can be a disadvantage, particularly when learning in unfamiliar domains. For instance, students who misinterpret a problem statement will most likely form an invalid mental model of the problem which they may find difficult to correct. A prior think-aloud study that used an intervention focused on this initial step in the problem-solving process showed promising results at a small scale. In this paper, we report on a large (n = 976) controlled experiment to examine the scalability of the aforementioned study. We measure the effect on novice programmer performance of metacognitive scaffolding that requires deliberate reflection on the problem statement before any code is written. Although control and experimental groups exhibited similar completion rates overall, students in the experimental group encountered significantly fewer errors that related to the formation of an incorrect mental model of the problem. These results hold promise in moving towards more reliable methods for developing metacognitive awareness in novice programmers.