Evaluation of “Out-of-the-Box” textbook technology supplements on student learning

As online learning becomes more prevalent in higher education, faculty are likely to increasingly turn to textbook technology supplements (TTSs) as a tool to enhance student learning outside the classroom. In 3 experiments, we tested whether using TTS multiple-choice quizzes (Experiments 1 and 2) or essay responses (Experiment 2) improved student learning as measured by publisher-provided multiple choice assessments administered in class. In Experiment 1 (N = 75), using a between-subjects design, we found no significant difference in performance for either in-class quizzes, p = .88, or exam performance, p = .79, as a function of whether students were required to use a TTSs or not. In Experiment 2 (N = 173), using a within-subjects design, we found a small but significant (3.5%) benefit to student learning on content for which they completed online quizzes (p = .002, partial η2 = .06). Experiment 3 (N = 90), again using a within-subjects design, found no significant benefit to learning when students were required to write essay responses to online questions (p = .13, partial η2 = .03). We suggest that the behaviors faculty intend to promote with the use of such tools (e.g., repeated practice, effortful attention, etc.) may not match the behaviors that these types of tools reinforce. They may serve as vehicles for engaged learning practices for some students, but for others, they may represent another system that must be “worked” to satisfy a requirement or get a good score.