The efficacy of well-designed instructional videos for STEM learning is largely reliant on how actively students cognitively engage with them. Students' ability to actively engage with videos likely depends upon individual characteristics like their prior knowledge. In this study, we investigated how digital trace data could be used as indicators of students' cognitive engagement with instructional videos, how such engagement predicted learning, and how prior knowledge moderated that relationship. One hundred twenty-eight biology undergraduate students learned with a series of instructional videos and took a biology unit exam one week later. We conducted sequence mining on the digital events of students' video-watching behaviors to capture the most commonly occurring sequences. Twenty-six sequences emerged and were aggregated into four groups indicative of cognitive engagement: repeated scrubbing, speed watching, extended scrubbing, and rewinding. Results indicated more active engagement via speed watching and rewinding behaviors positively predicted unit exam scores, but only for students with lower prior knowledge. These findings suggest that the ways students cognitively engage with videos predict how they will learn from them, that these relations are dependent upon their prior knowledge, and that researchers can measure students’ cognitive engagement with instructional videos via mining digital log data. This research emphasizes the importance of active cognitive engagement with video interface tools and the need for students to accurately calibrate their learning behaviors in relation to their prior knowledge when learning from videos.

Online learning occurs across a multidimensional ecosystem. Adolescents can leverage the affordances of each dimension, while also encountering challenges when learning online. Adolescents are not digital natives, and they struggle to learn online due to distractions and bias. Online learning's affordances include rich learning content across multiple representations. These environments can be personalized to increase adolescents' motivation and academic identities, increasing their achievement. Individualized online environments can support historically marginalized adolescents' achievement, increasing equity in education. We discuss the skills and dispositions adolescents must develop to manage their learning online and implications for the deployment of online learning environments for adolescents.

Even highly motivated undergraduates drift off their STEM career pathways. In large introductory STEM classes, instructors struggle to identify and support these students. To address these issues, we developed co-redesign methods in partnership with disciplinary experts to create high-structure STEM courses that better support students and produce informative digital event data. To those data, we applied theory- and context-relevant labels to reflect active and self-regulated learning processes involving LMS-hosted course materials, formative assessments, and help-seeking tools. We illustrate the predictive benefits of this process across two cycles of model creation and reapplication. In cycle 1, we used theory-relevant features from 3 weeks of data to inform a prediction model that accurately identified struggling students and sustained its accuracy when reapplied in future semesters. In cycle 2, we refit a model with temporally contextualized features that achieved superior accuracy using data from just two class meetings. This modelling approach can produce durable learning analytics solutions that afford scaled and sustained prediction and intervention opportunities that involve explainable artificial intelligence products. Those same products that inform prediction can also guide intervention approaches and inform future instructional design and delivery.Practitioner notesWhat is already known about this topicWhat this paper addsImplications for practice and/or policy Learning analytics includes an evolving collection of methods for tracing and understanding student learning through their engagements with learning technologies. Prediction models based on demographic data can perpetuate systemic biases. Prediction models based on behavioural event data can produce accurate predictions of academic success, and validation efforts can enrich those data to reflect students' self-regulated learning processes within learning tasks. Learning analytics can be successfully applied to predict performance in an authentic postsecondary STEM context, and the use of context and theory as guides for feature engineering can ensure sustained predictive accuracy upon reapplication. The consistent types of learning resources and cyclical nature of their provisioning from lesson to lesson are hallmarks of high-structure active learning designs that are known to benefit learners. These designs also provide opportunities for observing and modelling contextually grounded, theory-aligned and temporally positioned learning events that informed prediction models that accurately classified students upon initial and later reapplications in subsequent semesters. Co-design relationships where researchers and instructors work together toward pedagogical implementation and course instrumentation are essential to developing unique insights for feature engineering and producing explainable artificial intelligence approaches to predictive modelling. High-structure course designs can scaffold student engagement with course materials to make learning more effective and products of feature engineering more explainable. Learning analytics initiatives can avoid perpetuation of systemic biases when methods prioritize theory-informed behavioural data that reflect learning processes, sensitivity to instructional context and development of explainable predictors of success rather than relying on students' demographic characteristics as predictors. Prioritizing behaviours as predictors improves explainability in ways that can inform the redesign of courses and design of learning supports, which further informs the refinement of learning theories and their applications.

Undergraduates enrolled in large, active learning courses must self-regulate their learning (self-regulated learning [SRL]) by appraising tasks, making plans, setting goals, and enacting and monitoring strategies. SRL researchers have relied on self-report and learner-mediated methods during academic tasks studied in laboratories and now collect digital event data when learners engage with technology-based tools in classrooms. Inferring SRL processes from digital events and testing their validity is challenging. We aligned digital and verbal SRL event data to validate digital events as traces of SRL and used them to predict achievement in lab and course settings. In Study 1, we sampled a learning task from a biology course into a laboratory setting. Enrolled students (N = 48) completed the lesson using digital resources (e.g., online textbook, course site) while thinking aloud weeks before it was taught in class. Analyses confirmed that 10 digital events reliably co-occurred >= 70% of the time with verbalized task definition and strategy use macroprocesses. Some digital events co-occurred with multiple verbalized SRL macroprocesses. Variance in occurrence of validated digital events was limited in lab sessions, and they explained statistically nonsignificant variance in learners' performance on lesson quizzes. In Study 2, lesson-specific digital event data from learners (N = 307) enrolled in the course (but not in Study 1) predicted performance on lesson-specific exam items, final exams, and course grades. Validated digital events also predicted final exam and course grades in the next semester (N = 432). Digital events can be validated to reflect SRL processes and scaled to explain achievement in naturalistic undergraduate education settings.

The affordances of computer-based learning environments make them powerful tools for conveying information in higher education. However, to most effectively use these environments, students must be adept at self-regulating their learning. This self-regulation is effortful, including a myriad of processes, including defining tasks, making plans, using and monitoring the efficacy of high-quality learning strategies, and reflecting on the learning process and outcomes. Therefore, higher education instructors and course designers should design computer-based learning environments to ease learning and free up mental resources for self-regulation. This chapter describes how design principles from the cognitive theory of multimedia learning can facilitate learning in computer-based learning environments and promote self-regulated learning. Examples of the multimedia, personalization, and generative activity principles are presented to show how the cognitive theory of multimedia learning can guide design and promote students’ selection, organization, and integration of content, resulting in better understanding and more mental resources available for self-regulated learning and the deeper learning it can afford.

Well-designed instructional videos are powerful tools for helping students learn and prompting students to use generative strategies while learning from videos further bolsters their effectiveness. However, little is known about how individual differences in motivational factors, such as achievement goals, relate to how students learn within multimedia environments that include instructional videos and generative strategies. Therefore, in this study, we explored how achievement goals predicted undergraduate students' behaviors when learning with instructional videos that required students to answer practice questions between videos, as well as how those activities predicted subsequent unit exam performance one week later. Additionally, we tested the best measurement models for modeling achievement goals between traditional confirmatory factor analysis and bifactor confirmatory factor analysis. The bifactor model fit our data best and was used for all subsequent analyses. Results indicated that stronger mastery goal endorsement predicted performance on the practice questions in the multimedia learning environment, which in turn positively predicted unit exam performance. In addition, students' time spent watching videos positively predicted practice question performance. Taken together, this research emphasizes the availing role of adaptive motivations, like mastery goals, in learning from instructional videos that prompt the use of generative learning strategies.

Undergraduate science, technology, engineering, and mathematics (STEM) students' motivations have a strong influence on whether and how they will persist through challenging coursework and into STEM careers. Proper conceptualization and measurement of motivation constructs, such as students' expectancies and perceptions of value and cost (i.e., expectancy value theory [EVT]) and their goals (i.e., achievement goal theory [AGT]), are necessary to understand and enhance STEM persistence and success. Research findings suggest the importance of exploring multiple measurement models for motivation constructs, including traditional confirmatory factor analysis, exploratory structural equation models (ESEM), and bifactor models, but more research is needed to determine whether the same model fits best across time and context. As such, we measured undergraduate biology students' EVT and AGT motivations and investigated which measurement model best fit the data, and whether measurement invariance held, across three semesters. Having determined the best-fitting measurement model and type of invariance, we used scores from the best performing model to predict biology achievement. Measurement results indicated a bifactor-ESEM model had the best data-model fit for EVT and an ESEM model had the best data-model fit for AGT, with evidence of measurement invariance across semesters. Motivation factors, in particular attainment value and subjective task value, predicted small yet statistically significant amounts of variance in biology course outcomes each semester. Our findings provide support for using modern measurement models to capture students' STEM motivations and potentially refine conceptualizations of them. Such future research will enhance educators' ability to benevolently monitor and support students' motivation, and enhance STEM performance and career success.

Undergraduate STEM lecture courses enroll hundreds who must master declarative, conceptual, and applied learning objectives. To support them, instructors have turned to active learning designs that require students to engage in self-regulated learning (SRL). Undergraduates struggle with SRL, and universities provide courses, workshops, and digital training to scaffold SRL skill development and enactment. We examined two theory-aligned designs of digital skill trainings that scaffold SRL and how students’ demonstration of metacognitive knowledge of learning skills predicted exam performance in biology courses where training took place. In Study 1, students’ ( n = 49) responses to training activities were scored for quality and summed by training topic and level of understanding. Behavioral and environmental regulation knowledge predicted midterm and final exam grades; knowledge of SRL processes did not. Declarative and conceptual levels of skill-mastery predicted exam performance; application-level knowledge did not. When modeled by topic at each level of understanding, declarative knowledge of behavioral and environmental regulation and conceptual knowledge of cognitive strategies predicted final exam performance. In Study 2 (n = 62), knowledge demonstrated during a redesigned video-based multimedia version of behavioral and environmental regulation again predicted biology exam performance. Across studies, performance on training activities designed in alignment with skill-training models predicted course performances and predictions were sustained in a redesign prioritizing learning efficiency. Training learners’ SRL skills –and specifically cognitive strategies and environmental regulation– benefited their later biology course performances across studies, which demonstrate the value of providing brief, digital activities to develop learning skills. Ongoing refinement to materials designed to develop metacognitive processing and learners’ ability to apply skills in new contexts can increase benefits.

This study explored whether different types of instructional visuals—knowledge maps and pictorial illustrations—encourage students to focus on specific types of conceptual relationships during learning. Undergraduates (n = 134) studied a text lesson on the human nervous system accompanied by maps (text-with-maps group), illustrations (text-with-illustrations group), or no visuals (text-only group). Then all students orally explained what they learned as if they were teaching a peer. The text-with-maps group generated more hierarchical relationships than the other two groups, and both visual groups generated more temporal relationships than the text-only group. The groups did not significantly differ in the number of structural relationships generated. On a subsequent post-test, only the text-with-maps group significantly outperformed the text-only group, and the two visual groups did not significantly differ from each other. These findings highlight how different visuals affect the types of relationships students focus on when learning from the same text.

This study explored ways to foster generative learning during a narrated video lesson about the human kidney. In a 2 × 3 between-subjects design, 196 college students were randomly assigned to a video format condition and a learning strategy condition. Students listened to oral explanations from the instructor as they viewed either a series of static diagrams (static visuals) or the same diagrams dynamically drawn on the screen without the instructor visible (instructor-generated visuals). After each part of the lesson, students either wrote verbal explanations (explain) or created drawings (draw), or they rewatched that part of the lesson (rewatch). All students then completed retention and transfer tests on the material. Results indicated a significant main effect of learning strategy for retention and transfer: the explain group significantly outperformed the draw group (retention: d = .60; transfer: d = .67) and the rewatch group (retention: d = .58; transfer d = .87). There was also a significant video format by learning strategy interaction for transfer: explaining was most effective for students who observed instructor-generated visuals (ds > 1.0) rather than static visuals (ds < .5). These findings suggest that when learning from narrated video lessons with complex diagrams, students benefit most from viewing dynamically generated drawings and then verbally explaining what they learned. In contrast, creating drawings may not be appropriate for learning from diagram-heavy lessons. Overall, this study demonstrates the importance of aligning instructional methods with appropriate learning strategies. (PsycInfo Database Record (c) 2020 APA, all rights reserved)

This study tested 3 instructor presence features in learning from video lectures: dynamic drawings, eye contact with the camera, and instructor visibility. In 2 experiments, college students watched a video lecture about the human kidney, which consisted of a series of drawings and a spoken explanation from the instructor, and then took a written posttest assessing retention and transfer. In Experiment 1, students viewed a lesson consisting of a spoken explanation coordinated with static, already-produced drawings (static drawings group) or with drawings dynamically created by the instructor (dynamic drawings group), both without the instructor visible. In support of the dynamic drawings hypothesis, a t test indicated the dynamic drawings group significantly outperformed the static drawings group on the posttest (d = .54). In Experiment 2, students viewed 2 new versions of the kidney lesson, in which the instructor was visible on the screen and either did not provide eye contact with the camera (conventional whiteboard group) or did provide eye contact (transparent whiteboard group). In support of the social agency hypothesis, a t test indicated the transparent whiteboard group significantly outperformed the conventional whiteboard group on the posttest (d = .54). Finally, consistent with the instructor visibility hypothesis, analyses comparing the dynamic drawings group and the transparent whiteboard group indicated no significant differences in posttest performance. Overall, these findings suggest that learning from video lectures is enhanced by specific instructor presence features, such as instructor dynamic drawing and instructor eye contact, rather than by merely having the instructor visible on the screen.

This study tested the effects of implementing a narrative computer-based educational game within a middle-school math class. Gameplay consisted of navigating through a virtual spaceship and completing missions by periodically engaging in learning-by-teaching activities that involved helping an avatar solve math problems. In a pretest/posttest matched-groups design, 58 middle-school students either played the game for 10 hours over 4 days in place of their typical math instruction (game group), or they received conventional math instruction that consisted of a matched set of practice problems (control group). Contrary to our hypotheses, results from posttest measures indicated no significant differences in learning outcomes or motivation between the two groups. Importantly, supplementary observational data indicated that students in the game group spent much of their time during gameplay engaging in activities unrelated to the educational content of the game (e.g., navigating the virtual world) and only 20% of their time engaging in learning-by-teaching activities. These results highlight the importance of designing educational games that effectively balance features intended to entertain learners and features intended to promote learning. Implications for implementing educational games into classroom instruction are discussed.

This study tested whether creating drawings helps students generate higher-quality oral explanations during learning by teaching, thereby enhancing learning outcomes. 120 college students studied a scientific text about the human respiratory system. Students then either taught the material on video to a fictitious peer by orally explaining (explain-only), creating drawings (draw-only), or creating drawings while orally explaining (explain-and-draw). A control group of students spent the same amount of time restudying the lesson (restudy). One week later all students completed a posttest consisting of retention, transfer, and drawing questions. All three teaching conditions significantly outperformed the restudy condition on the posttest (d's ranging from.80 to 1.46). Critically, the explain-and-draw group also significantly outperformed the explain-only (d = .99) and the draw-only (d = .65) groups. Consistent with our primary hypotheses, the explain-and-draw group produced more elaborative oral explanations than the explain-only group, which partially explained the benefits of drawing while explaining on learning outcomes. Overall, this study demonstrates that drawing facilitates explaining and enhances the effectiveness of learning by teaching.