Social-psychological interventions have raised the learning and performance of students in rigorous efficacy trials. Yet, after they are distributed "in the wild" for students to self-administer, there has been little research following up on their translational effectiveness. We used cutting-edge educational technology to tailor, scale up, and track a previously-validated Strategic Resource Use intervention among 12,065 college students in 14 STEM and Economics classes. Students who self-administered this "Exam Playbook" benefitted by an average of 2.17 percentage points (i.e., a standardized effect size of 0.18), compared to non-users. This effect size was 1.65 percentage points when controlling for college entrance exam scores and 1.75 [-1.88] for adding [dropping] the Exam Playbook in stratified matching analyses. Average benefits differed in magnitude by the conduciveness of the class climate (including peer norms and incentives), gender, first-generation status, as well as how often and how early they used the intervention. These findings on how, when, and who naturally adopts these resources address a need to improve prediction, translation, and scalability of social-psychological intervention benefits.

Large courses act as gateways for college students and often have poor outcomes, particularly in STEM fields where the pace of improvement has been glacial. Students encounter barriers to persistence like low grades, competitive cultures, and a lack of motivation and belonging. Tailored technology systems offer one promising path forward. In this observational study, we report on the use of one such system, called ECoach, that provides students resources based on their psychosocial profile, performance metrics, and pattern of ECoach usage. We investigated ECoach efficacy in five courses enrolling 3,599 students using a clustering method to group users by engagement level and subsequent regression analyses. We present results showing significant positive relationships with small effect sizes between ECoach engagement and final course grade as well as grade anomaly, a performance measure that takes into account prior course grades. The courses with the strongest relationship between ECoach engagement and performance offered nominal extra credit incentives yet show improved grades well above this “investment” from instructors. Such small incentives may act as a catalyst that spurs deeper engagement with the platform. The impact of specific ECoach features and areas for future study are discussed.

For nearly a century, pre-college standardized test scores and undergraduate letter grades have been de facto industry standard measures of achievement in US higher education. We examine a sample of millions of grades and a half million pre-college test scores earned by undergraduates between 2006 and 2019 at a large public research university that became increasingly selective, in terms of test scores of matriculated students, over that time. A persistent, moderate correlation between test score and grades within the period motivates us to employ a simple importance sampling model to address the question, “How much is increased selectivity driving up campus grades?”. Of the overall 0.213 rise in mean undergraduate grade points over the thirteen-year period, we find that nearly half, 0.098 ± 0.004, can be ascribed to increased selectivity. The fraction is higher, nearly 70%, in engineering, business and natural science subjects. Removing selectivity’s influence to surface curricular-related grade inflation within academic domains, we find a factor four range, from a low of ∼ 0.05 in business and engineering to a high of 0.18 in the humanities, over the thirteen year period.

Gameful pedagogy is a novel approach to teaching that has emerged in the past decade that emphasizes intentionally designing curricula to support student motivation. In this study we investigate how five gameful courses at a large R1 university have impacted students when analyzed with an eye towards equity: are men vs women, underrepresented minorities vs majority students, and first-generation students vs traditional students able to achieve similar amounts of success in gameful courses? Results show that for both men and minority students, there is evidence to suggest they are underachieving in the courses studied as compared to women and majority students, but when we control for prior academic performance these trends disappear. For first-generation students, we see conflicting evidence, with cases of both under and over-achievement present. We again see these discrepancies disappear when we control for prior performance.