Empirical Synthesis of the Effect of Standard Error of Measurement on Decisions Made within Brief Experimental Analyses of Reading Fluency

Intervention researchers often use curriculum-based measurement of reading fluency (CBM-R) with a brief experimental analysis (BEA) to identify an effective intervention for individual students. The current study synthesized data from 22 studies that used CBM-R data within a BEA by computing the standard error of measure (SEM) for the median data point from the baseline and intervention data. The median CBM-R score from the intervention that the authors of each study identified as most effective fell within the SEM (68% confidence interval) of the baseline data approximately 30% of the time, but the ranges for the two author-identified most effective interventions overlapped over 75% of the time. Extended analyses were consistent with the BEA results for approximately three-fourths of the instances after considering the SEM of the baseline and intervention phases. Using matched passages did not improve the overlap of the ranges, but there was less overlap when the study used three data points per condition. Results emphasize the importance of considering SEM of CBM-R data when comparing interventions within a BEA. Further implications for practice and future research are included.