There is a growing effort to inform teachers about the effects of various educational practices; however, the uncertainty associated with these effects is often very large and rarely reported. As a result, teachers may be unaware that the actual effects of interventions can often deviate substantially from what is reported. Should we communicate the uncertainty of educational effects to teachers? Doing so could support their decision-making, yet research from other fields (e.g., in risk communication) suggests that reporting uncertainty can be perceived negatively. We conducted two pre-registered studies (total N = 995) to examine this question. Reporting uncertainty did not impact the perceived understandability of the information presented but decreased the perceived effectiveness of an intervention, suggesting that reporting uncertainty may influence teachers' decisions to implement an educational practice in their classrooms. However, we also found that reporting uncertainty leads to a decrease in trust in those who communicated the information, a particularly worrying observation, given the ubiquity and magnitude of uncertainty in educational research. We discuss the implications of these findings for research communicators, research evaluators, and initial teacher education.
Improving early-grade reading in linguistically diverse, resource-constrained settings remains challenging. This study evaluated a nationwide bilingual reading intervention in Ghana that provided reading materials, teacher guides, sequenced training, and ongoing coaching. Using a randomized trial with 3,559 second- and third-grade students in 198 schools, we examined program impacts and how student, teacher, and school languages moderate these effects. The intervention improved reading outcomes across letter-sound knowledge, decoding, fluency, and comprehension, yielding main effect sizes of 0.09 to 0.43 standard deviations for Ghanaian languages and 0.12 to 0.37 standard deviations for English. These gains were amplified when students and teachers shared the same first language for classroom oral exchanges. Effect sizes for students in concordant student-teacher pairs were 0.17 to 0.42 standard deviations higher than those for students in discordant pairs. These results show that large-scale reading improvements are achievable when classroom instruction is strengthened, and language-of-instruction policies account for teacher-student language concordance, not just school- or district-level assignments.
Randomized controlled trials (RCTs) produce impact evidence with high internal validity but uncertain external validity. External validity bias has been defined as the expected difference between the average impact in the sample and the average impact in the population. This study estimated the external validity bias from several site selection methods by simulating hypothetical RCTs of the Head Start program in which some selected sites decline to participate. Three main findings emerged from the analysis. First, purposive site selection consistently produced biased impact estimates that varied in magnitude based on the outcome examined, the factors used to select sites, and the factors that influenced site decisions to participate. Second, simple random site selection yielded less external validity bias than purposive site selection under most tested conditions. Third, stratified random site selection yielded virtually no external validity bias, but the results likely overstate the method's performance when data on impact moderators are unavailable before sites are selected. These findings offer lessons on how to select sites in future RCTs to minimize external validity bias.
The study presents evidence from a multi-arm cluster randomized evaluation examining the impact and cost-effectiveness of three teacher professional development (TPD) components implemented within a large-scale early grade reading intervention in Pakistan. A sample of 200 schools, 185 teachers and 3068 and Grade 1 students was randomly assigned to receive a full TPD package -including face-to-face training (FtF), teacher inquiry groups (TIGS), and school support visits (SSVs)- or one of three reduced variants omitting one component. Teacher instructional practices were measured using a Teacher Classroom Observation tool, student literacy using the Early Grade Reading Assessment (EGRA), and cost using an ingredients-based approach. All TPD components improved instructional practices; however, only SSVs generated statistically significant gains in reading outcomes (0.23 standard deviations in overall literacy and 0.30 in reading comprehension). Although SSVs were the most expensive component (US$6 per child) they were also the most cost-effective. FtF (US$3 per child) and TIGs (US$5 per child) improved instruction but did not yield detectable literacy gains. Findings provide policy-relevant evidence on cost effective TPD in conflict affected settings.
