Gamified educational technologies are often used in education to teach concepts and better motivate students, but questions remain on how to present feedback that supports persistence-related replay learning behaviors. We investigated how providing reward- and error-based feedback related to replay behaviors and problem-solving strategies in From Here to There! (FH2T), an algebra-focused educational technology that utilizes Graspable Math technology and research infrastructure. 1,011 seventh-grade students played nine 30-minute sessions of FH2T. Multilevel logistic regressions examined whether: 1) the number of performance-contingent rewards earned (one to three-star rewards based on solution efficiency) or making at least one error on their first problem-solving attempt predicted students’ choice to replay problems, and 2) replaying was associated with better problem-solving efficiency. Earning two stars was associated with a higher likelihood of replaying compared to one or three stars. Students who replayed problems were more likely to improve their problem-solving efficiency and achieve optimal efficiency on both the replayed problem and the following problem compared to students who did not replay. The option to replay problems without penalty, alongside performance-contingent reward-based feedback, may increase motivation and encourage persistence-related behaviors that are positively associated with learning outcomes. These findings have implications for designing game-like elements for learning and motivation in educational technology and using these contexts as a research infrastructure.

Perceptual learning theory suggests that perceptual grouping in mathematical expressions can direct students' attention toward specific parts of problems, thus impacting their mathematical reasoning. Using in-lab eye tracking and a sample of 85 undergraduates from a STEM-focused university, we investigated how higher-order operator position (HOO; i.e., multiplication/division operators and the presence of superfluous brackets impacted students' time to first fixation to the HOO, response time, and percent of correct responses). Students solved order-of-operations problems presented in six ways (3 HOO positions x presence of brackets). We found that HOO position and presence of superfluous brackets had separate and combined impacts on calculating arithmetic expressions. Superfluous brackets most influenced undergraduates' performance when higher-order operators were located in the center of mathematical expressions. Implications for learning and future directions are discussed about observing eye movements and gaining insights into students' processes when solving arithmetic expressions. Using eye tracking, we examined how the position of higher-order operators (i.e., multiplication and division signs) and the presence of superfluous brackets (i.e., brackets that do not change the order of calculations when included) impacted undergraduate students' attentional processing and performance when solving arithmetic expressions. The results show that including superfluous brackets was most impactful when higher-order operators were in the center of mathematical expressions. Findings indicate how subtle changes in mathematical expressions can influence arithmetic performance.

The impact of educational programs on student learning is contingent upon the quality and fidelity of their implementations. Yet, the most reliable method of implementation monitoring, direct observation, may be infeasible for large-scale efficacy studies. Here, we use log-file data from an efficacy study assessing the impact of gamified learning technologies on algebraic knowledge to measure the fidelity of implementations remotely within and across multiple programs. We employ exploratory-data-analysis to identify measures of fidelity, multilevel models to evaluate which levels of implementation contexts (i.e. classrooms, teachers, students) were associated with variance in fidelity metrics, and latent profile analysis to identify high and low-fidelity contexts. These analyses illustrate how log-file data may be used to remotely monitor and evaluate the implementations of online learning platforms.

Download This Paper Open PDF in Browser Add Paper to My Library Share: Permalink Using these links will ensure access to this page indefinitely Copy URL Copy DOI

Prior research has shown that game-based learning tools, such as DragonBox 12+, support algebraic understanding and that students' in-game progress positively predicts their later performance. Using data from 253 seventh-graders (12-13 years old) who played DragonBox as a part of technology intervention, we examined (a) the relations between students' progress within DragonBox and their algebraic knowledge and general mathematics achievement, (b) the moderating effects of students' prior performance on these relations and (c) the potential factors associated with students' in-game progress. Among students with higher prior algebraic knowledge, higher in-game progress was related to higher algebraic knowledge after the intervention. Higher in-game progress was also associated with higher end-of-year mathematics achievement, and this association was stronger among students with lower prior mathematics achievement. Students' demographic characteristics, prior knowledge and prior achievement did not significantly predict in-game progress beyond the number of intervention sessions students completed. These findings advance research on how, for whom and in what contexts game-based interventions, such as DragonBox, support mathematical learning and have implications for practice using game-based technologies to supplement instruction. Practitioner notesWhat is already known about this topicDragonBox 12+ may support students' understanding of algebra but the findings are mixed.Students who solve more problems within math games tend to show higher performance after gameplay.Students' engagement with mathematics is often related to their prior math performance. What this paper addsFor students with higher prior algebraic knowledge, solving more problems in DragonBox 12+ is related to higher algebraic performance after gameplay.Students who make more in-game progress also have higher mathematics achievement, especially for students with lower prior achievement.Students who spend more time playing DragonBox 12+ make more in-game progress; their demographic, prior knowledge and prior achievement are not related to in-game progress. Implications for practice and/or policyDragonBox 12+ can be beneficial as a supplement to algebra instruction for students with some understanding of algebra.DragonBox 12+ can engage students with mathematics across achievement levels.Dedicating time and encouraging students to play DragonBox 12+ may help them make more in-game progress, and in turn, support math learning.

BackgroundPrior work has shown that middle school students struggle with algebra and that game-based educational technologies, such as DragonBox and From Here to There!, are effective at improving students' algebraic performance. However, it remains unclear which aspects of algebraic knowledge shift as a result of playing these games and what game design features support algebraic learning. ObjectiveUsing the data from a randomized controlled trial conducted in the 2020-2021 academic year, we (a) examined students' relative performance on assessment items measuring conceptual knowledge, procedural knowledge and procedural flexibility in algebraic equation solving, and (b) identified changes in these aspects of algebraic knowledge after playing DragonBox or From Here to There!. MethodsEight hundred eighty-seven seventh-graders were randomly assigned to playing DragonBox or From Here to There! for nine 30-min sessions throughout the school year. Students also completed a pretest, midtest and posttest measuring their algebraic knowledge. Results and ConclusionFirst, prior to the intervention, students scored the highest on procedural knowledge and lowest on conceptual knowledge. Second, students significantly improved and maintained learning gains on conceptual knowledge throughout the intervention. Their performance on procedural knowledge and procedural flexibility items increased at midtest but decreased at posttest. Third, the pattern of results was consistent for students in the DragonBox and From Here to There! conditions, suggesting that both games may support students' conceptual understanding of algebra. The findings have implications for research and practice on supporting algebraic learning through game-based technologies during and beyond educational disruptions.

This symposium brings together eleven projects across three continents to examine notions of disruption in educational research.Historically, notions of disruption have pointed to the ways research leverages innovation and transformative practice.However, amid global pandemic and intersecting unrest, the authors in this session recognize the need for deeper conversation across contexts to understand the ways that educational research in the learning sciences can leverage disruption toward transformational learning.To respond to this year's call for building and sustaining knowledge in community, we seek to foster conversation about the ways that designed for and encountered disruptions act as opportunities for critical reflection and new kinds of engagement in educational research.Together, we examine various notions of disruption as they exist in our disparate work.We seek to provoke meaningful conversations about the ways educational research can embody the contemporary realities of learning with and toward disruption.

As evidence grows supporting the importance of non-cognitive factors in learning, computer-assisted learning platforms increasingly incorporate non-academic interventions to influence student learning and learning related-behaviors. Non-cognitive interventions often attempt to influence students’ mindset, motivation, or metacognitive reflection to impact learning behaviors and outcomes. In the current paper, we analyze data from five experiments, involving seven treatment conditions embedded in mastery-based learning activities hosted on a computer-assisted learning platform focused on middle school mathematics. Each treatment condition embodied a specific non-cognitive theoretical perspective. Over seven school years, 20,472 students participated in the experiments. We estimated the effects of each treatment condition on students’ response time, hint usage, likelihood of mastering knowledge components, learning efficiency, and post-tests performance. Our analyses reveal a mix of both positive and negative treatment effects on student learning behaviors and performance. Few interventions impacted learning as assessed by the post-tests. These findings highlight the difficulty in positively influencing student learning behaviors and outcomes using non-cognitive interventions.

The current study investigated the effectiveness of three distinct educational technologies-two game-based applications (From Here to There and DragonBox 12+) and two modes of online problem sets in ASSISTments (an Immediate Feedback condition and an Active Control condition with no immediate feedback) on Grade 7 students' algebraic knowledge. More than 3,600 Grade 7 students across nine in-person and one virtual schools within the same district were randomly assigned to one of the four conditions. Students received nine 30-minute intervention sessions from September 2020 to March 2021. Hierarchical linear modeling analyses of the final analytic sample (N = 1,850) showed significantly higher posttest scores for students who used From Here to There and DragonBox 12+ compared to the Active Control condition. No significant difference was found for the Immediate Feedback condition. The findings have implications for understanding how game-based applications can affect algebraic understanding, even within pandemic pressures on learning.

Using data from 183 U.S. seventh graders, we examined whether students' DragonBox12+ progress related to their later mathematics achievement, and whether students' prior achievement moderated this relation.Higher in-game progress was associated with higher end-of-year mathematics achievement, and this association was stronger among students with lower prior mathematics achievement.These findings advance research on how and for whom DragonBox12+ supports mathematical learning, and have implications for practices using game-based technologies to supplement instruction.

Computer-assisted learning platforms (CALPS) increasingly include gamified elements to improve student outcomes by enhancing their engagement with content. Although evidence exists that gamified programs increase engagement and learning outcomes, there is little causal research on what programmatic mechanisms drive the effect between engagement and learning. In the following paper, we explore this relationship through a method of causal moderation known as fully latent principal stratification. Using data from a large-scale randomized control trial assessing gamified and traditional CALP systems' effects on algebraic knowledge, we estimate the impact of using the gamified CALP on students who engage with one of its key gamification elements---replaying a problem after a suboptimal attempt. The gamified CALP asks students to manipulate algebraic expressions from start to goal states and provides feedback based on the efficiency of these manipulations, allowing students to replay the problems when their efficiency can be improved. We find that the effect of gamification is greater for students with a higher propensity to replay problems. This finding suggests that gamification elements that provide students with opportunities to retry problems are driving the game's efficacy and provide evidence for a scalable mechanism of gamification that can improve students' learning.

Cognitive, numeracy, and motivational factors have been implicated in math achievement. However, few studies have investigated these factors simultaneously and in middle childhood, limiting our understanding of the relative contributions of these factors during an important developmental period. The current study investigated how one numeracy, four cognitive, and three motivational factors predicted math performance in 525 third, 771 fourth, and 465 fifth grade students, and whether these relations changed by grade and the type of math measured. Participants completed the California Standards Test (CST; Educational Testing Service, 2014) in Spring 2012 and Spring 2013 to assess math performance, and a battery of numeracy, cognitive, and motivational measures in Spring 2013. Separate multilevel regressions predicting 2013 CST performance were conducted for each grade, and z scores were used to compare the coefficients between grades. Results showed that updating and math expectancy were related to broad-level math performance for all grades. Further, numeracy and shifting were most predictive of math performance for younger grades, whereas math expectancy was most predictive for older grades across a range of math content. The results suggest that numeracy, cognitive, and motivational factors all relate to math performance when these three types of factors are considered, with numeracy, updating, shifting, and math expectancy showing the most reliable relations depending on grade. The relations between factors and math types may also have implications for interventions aiming to improve specific types of math.

The ability to accurately measure academic motivation is important to its value as a predictive variable for learning, achievement, and other outcomes. Although measures of motivation are frequently subject to quan-titative validation (e.g., Appleton, Ntoumanis, Quested, Viladrich, & Duda, 2016; Gagne ' et al., 2015; Pekrun, Goetz, Frenzel, Barchfeld, & Perry, 2011), the establishment of cognitive validity is more rare. By conducting cognitive interviews with a sample of elementary-aged children, we explored the cognitive validity of a novel motivation (expectancy-value and academic emotions) survey embedded in an educational technology. Children were largely able to accurately interpret questions, elaborate on their reasoning for answers, and choose answers congruent with those reasons. Challenges to cognitive validity fell under varied and underdeveloped in-terpretations of expectancy-value concepts; misunderstandings related to available response choices; and dis-crepancies between younger and older children's abilities to judge their perceived competencies and values. Insights from these interviews can be applied to interpretation of the immediate survey, but also to design and interpretation of motivation surveys beyond the current measure.

Background Large achievement and motivation gaps exist in science between students from higher and lower socioeconomic status (SES) backgrounds. Middle and high school are an important time to address these disparities, as science motivation typically declines for all students at this time, leading to particularly low science interest and achievement for lower SES students on average when the gaps are left unaddressed. Students’ control over their free time also increases at this time, providing opportunities for optional science experiences that may improve science attitudes and skills to combat these achievement and motivation gaps. Using a longitudinal dataset of 2252 middle and high school students from two regions in the USA, we investigate (1) disparities between higher and lower SES students in participation in optional summer science experiences and post-summer science attitudes and skills; (2) whether the child and family characteristics that predict participation in home-related, nature-related, and STEM camp experiences in the summer differ for higher and lower SES students; and (3) how participation in these types of optional summer science experiences contribute to post-summer science attitudes and skills when controlling for self-selection biases. Results Higher SES students reported greater participation in optional summer science experiences and higher post-summer science attitudes and sensemaking skills. Fascination for science was more important for participation in home-related and nature-related experiences for higher SES participants, whereas science competency beliefs were more important for lower SES participants. For STEM camp experiences, higher SES participants with higher competency beliefs and lower SES participants with lower scientific sensemaking skills were more likely to participate. After controlling for self-selection biases that may influence participation in these experiences, we found that home-related and nature-related experiences had a positive impact on students’ attitudes toward science. Conclusions Our findings suggest two pathways for increasing participation in optional summer science experiences for higher SES and lower SES students. Specifically, it may be helpful to support interest in science for higher SES students and competency beliefs for lower SES students. Greater participation in home-related and nature-related summer science experiences can also increase science attitudes during middle and high school.

Self-beliefs are important determinants of student choice and success (Wigfield & Eccles, 2000) and are informed by student educational experiences, such as prior success with a task (Bandura, 1986). The potential for Computer-Based Interventions as self-belief-supporting learning environments is examined in this study, focusing on the mathematics software, Spatial Temporal (ST) Math. ST Math includes elements theorized to support student self-beliefs, including informative feedback and a self-pacing structure. Using a randomized control trial, we find that students who play ST Math have higher mathematics self-beliefs than their control counterparts, and that ST Math operates through self-beliefs to positively influence achievement. ST Math's impact on student self-beliefs is strongest for those students who had lower mathematics achievement scores.

Science learning is most often examined within formal education contexts, even though students spend more of their lives outside of the classroom. Students may interact with scientific phenomena during these out-of-school experiences, providing additional opportunities for learning to take place. Prior studies have found that optional science experiences have positive effects on science knowledge and attitudes. However, these studies do not always account for initial differences between students who are able to participate in many optional experiences and those who cannot. Moreover, many studies focus on high-quality science programs, which may not be representative of the average out-of-school experience. Using a longitudinal dataset of 3,700 6th and 8th grade students in urban and suburban schools from two regions in the United States, the current study investigates the effects of typically occurring optional science experiences during the school year on the development of science knowledge and attitudes. Using propensity score matching, we matched participating and nonparticipating students for characteristics that drive self-selection into these experiences, then analyzed separately for effects of school-related versus home-related science learning experiences. Stable patterns across analytic models reveal growth in science attitudes with both school- and home-related science experiences, but a greater relative decline in science knowledge with school-related science experiences. Thus, typically occurring optional science experiences can influence students’ attitudes and knowledge, but the effects can vary by the type of experience that students attend.

Playing action video games has previously been linked to improvements in attentional control, the ability to choose what to attend and what to ignore that relies on a frontoparietal network of the brain. Here we asked whether action video game training would impact a range of mathematical abilities that rely on similar brain regions. Twenty-four adults completed extensive cognitive testing before, after 25 hr, and after 40 hr of video game training. Half of the participants trained on an action video game, the other half trained on a nonaction video game. Action video game training yielded no significant improvements in foundational number-processing skills and attentional control in this study but some improvements on standardized assessments of complex mathematics. Thus, although action video game play is clearly not an intervention of choice when considering mathematical skills, the present study suggests its use as a recreational activity may support, albeit weakly, complex mathematical skills.