Assessing Educational Effectiveness: Policy Implications from Diverse Areas of Research

In England, school league tables - based on student scores on standardised tests - are used extensively by policymakers and parents as one of the key measures of secondary school effectiveness. Here, we briefly review the English contextual value added (E-CVA) model used to construct league tables of school effectiveness and juxtapose it with studies that focus on teacher effectiveness, other criteria of educational effectiveness and effectiveness at the university level. Methodologically, we consider the fragility of causal inferences based on psychometric and econometric value added models, and new multilevel models that take into account measurement and sampling error. Value added models largely ignore measurement error which systematically biases the outcomes, but both measurement and sampling error can be incorporated into doubly-latent models that integrate multilevel and latent variable approaches. The E-CVA model focuses on school effects and ignores teacher effects. However, research on both secondary schools and universities shows that there is little variance at the school or university level, but much more at the teacher level. E-CVA models consider test scores as the only indicator of effectiveness; there is little emphasis on other indicators of effectiveness, tests of the construct validity of test scores as a measure of effectiveness in relation to other indicators, or formative evaluations to improve effectiveness. Research on university students' ratings of teachers incorporates a broader perspective to construct validation: multidimensional factor structure, reliability/stability, relations with multiple criteria of effective teaching, and interventions to improve teaching. Both areas of research could learn from the other, and many strategies from the higher-education literature could be incorporated into the literature on educational effectiveness at the primary- and secondary-school levels.