This chapter focuses on using logic models as a way of framing mixed methods approaches, which have come to prominence in randomised controlled trials (RCTs). The framework and rationale of logic models motivates a principled eclecticism in our approach to mixed methods research (MMR). It justifies the combination of data collection methods and analyses in measuring and testing the process and outcomes of large-scale research, including RCTs. We summarise an approach to logic models developed in earlier work (Coldwell & Maxwell, 2018) and exemplify its application to a specific RCT, Integrating English, highlighting prompts that can determine an appropriate mix of data collection methods. From these and other applications, we consider issues in the future and challenges involved in applying this approach in MMR. The case is made for MMR exposing the strengths and limitations of logic models and for providing rich, action-oriented data on intervention-based research such as RCTs.

Discussions of randomised controlled trials (RCTs) in education that do not show an impact regularly focus on the intervention and how it failed to impact on expected measures, with typologies identifying persistent critical points of failure. This paper uses one such RCT—the Integrating English programme—to exemplify the application of a new model to explain failure in RCTs. To do so, the paper develops a set of categories of context drawing on the wider social evaluation field: backdrop, design, operation and interpretation. Thus, the paper exposes critical weak points in the commission and interpretation, as well as the implementation, of an RCT. Our aim is to work towards more robust evaluations by demonstrating that it is not simply the programme design, implementation and evaluation that can contribute to a lack of impact; there can be more fundamental system issues at play.

Discussions of randomised controlled trials (RCTs) in education that do not show an impact regularly focus on the intervention and how it failed to impact on expected measures, with typologies identifying persistent critical points of failure. This paper uses one such RCT—the Integrating English programme—to exemplify the application of a new model to explain failure in RCTs. To do so, the paper develops a set of categories of context drawing on the wider social evaluation field: backdrop, design, operation and interpretation. Thus, the paper exposes critical weak points in the commission and interpretation, as well as the implementation, of an RCT. Our aim is to work towards more robust evaluations by demonstrating that it is not simply the programme design, implementation and evaluation that can contribute to a lack of impact; there can be more fundamental system issues at play.

Models of research use in education tend to focus on specific elements of education systems or underplay the complexity of system change. Within other public policy areas, notably health, more work has been undertaken to integrate systems thinking when considering knowledge mobilisation and research use. In this paper, we survey public policy system change literature to develop a set of system dimensions. We use these to examine models relating to research use that are widely referenced in education. We then apply these dimensions to the work of the Education Endowment Foundation (EEF), the UK's What Works Centre for Education, which aims to support evidence-informed practice at all levels of the education system. We focus on its work to embed research-informed practices in regional school systems, through a case analysis of two 'scale-up campaigns' to mobilise evidence relating to the effective deployment of teaching assistants (educational support paraprofessionals). The findings highlight the value of using the system dimensions framework as a diagnostic tool to understand how to effect system change, highlighting the key role of brokerage and system leadership at different system levels; school-level capacity to implement change; and system relationships. Rationale for this studyContext and implications Implications for educational researchers and policymakersWhy the new findings matter Current approaches to supporting research use in education underplay the complexity of system change. Failure to acknowledge the complexity of research-use systems is likely to result in less-than-optimal approaches and interventions to improve research use. By applying a systems perspective, we explore how research use can be more effectively supported. Research use emerges as a myriad of interconnected 'moving parts' that need to function optimally and be aligned. Weakness in any area of the system, or interactions between system actors and activities across system levels, can potentially impede research use. A systems perspective, using the dimensions table presented in the paper, can be used to examine the functioning of existing systems and make informed decisions on where best to intervene to support practitioners' use of research. There is value in exploring multi-stranded mobilisation strategies that work together at different levels of the system e.g. school, regional policy, national.

This article focuses on the evaluation of the Children's Community programme, addressing the question: how can evaluators effectively address complex place-based system change initiatives, particularly those working across differing contexts? After considering alternatives, including what we describe as 'first-generation' linear, logic model-based approaches and second-generation Theory of Change approaches, we discuss the development of 'third-generation' systems-change evaluations and articulate a novel comparative maturity matrix approach, developed for the evaluation of Children's Communities with potential wider applicability for evaluation of complex systems change interventions. The article discusses the evaluation approach in depth and concludes with reflection on what learning points may be derived for use in subsequent evaluations of place-based and complex systems change initiatives.

This chapter addresses evidence-informed teaching in the English context. The chapter makes the case for considering England as having elements of both high and low social cohesion, with an increasingly narrow role for Local Authorities (districts) alongside the growth of more powerful but smaller Multi-Academy Trusts. Within the context of a highly regulated accountability regime, this places England in the hierarchist, with elements of fatalist, quadrant of the cohesion/regulation matrix. England has a well-developed infrastructure for supporting research use, including the Education Endowment Foundation (EEF), a very well-resourced charity acting as the What Works Centre for school education. Despite this, use of research evidence amongst teachers is low, and this has changed little since the mid-2010s. The chapter draws on institutional theory to explain this finding. The following explanations are provided: a lack of resources, coupled with a strong, politicised accountability system and a hollowed out middle tier to support schools, contributing to a lack of prioritisation amongst school leaders. However, England's well-developed infrastructure has enabled the EEF to play a significant and evidence-led role in supporting schools in the context of the COVID-19 pandemic. The chapter concludes by suggesting a series of suggestions to improve use of research in England. These include alignment of policy with research evidence; support for school leaders; work to explicitly link research to the evidence forms and supporting research brokerage.

Globally, there are ongoing problems with teacher retention, leading to a loss of experience and expertise. In policy and research, the emphasis is often on the professional development and retention of early career teachers, whereas teachers in later stages of their career are relatively under-represented. This article addresses this imbalance, reporting on a mixed methods scoping study that explores definitions of mid-career teachers in England and their retention and development, via a literature review, primary data collection and secondary analysis of data from the OECD's TALIS 2018 survey. We found that there is no agreed definition of mid-career teacher, relating to time in teaching, role and wider life circumstances and self-definition. Whatever definition is used, mid-career teachers are a heterogenous group, with varying needs, career plans and commitment to the profession. Whilst typically confident in their practice, their learning needs vary and are often experienced as unmet, especially for those looking for progression routes outside leadership and those with family commitments. This indicates that their potential for career development to benefit the profession may not be reached. The article concludes with suggestions for further study, policy and practice to improve understanding of this under-researched group.

Curriculum materials (schemes of work, lesson plans, etc.) play a complex and pivotal role in school and teacher practices. The adaptation and development of curriculum materials often constitute part of teacher professional development (PD) activities. However, compared with research examining the relationship between PD and teacher professional change, the role of curriculum materials in professional learning remains under-researched and under-theorised. We address this gap by applying a multi-perspectival approach to data from a PD programme in which teachers were supported to develop curriculum materials. We use an interconnected model to analyse the role of curriculum materials in catalysing change in individual teachers' practice. Our use of Boundary Theory proposes that curriculum material adoption is mediated by the solidity of boundaries between school practice and research findings, and Actor-Network Theory perspectives examine the assemblage of networked relations within and beyond schools that are entangled in curriculum materials. We highlight how combining linear and non-linear perspectives may contribute to improved understanding of the complexity of supporting teachers' learning and use our analyses to outline implications of using curriculum materials in teacher professional development.

ABSTRACT Since the 1990s, there have been repeated calls for the systematic use of randomised controlled trials (RCTs) to inform educational decision-making. The advent of the Education Endowment Foundation (EEF) – described as England’s What Works Centre for Education – in 2011 has made this a reality in England: by 2020, over a third of English schools were involved in such trials. Despite much debate about the value and role of RCTs, less attention has been paid to one specific effect of such trials, a phenomenon we refer to as interventionisation. This article uses two examples, focused on language and literacy education and teacher professional development, to demonstrate how increased use of trials may work to ‘interventionise’ education through channelling the focus of innovation and development to tightly structured interventions and generating a series of narrowing effects. It argues that a broad view of research and a diversity of methodologies is needed not only to generate rich understandings of educational practice but to develop and sustain educational provision that is fit for a dynamic world and which responds to the challenges and opportunities presented in complex educational contexts.

This tool for teachers is a product of the data collection and analysis, including interviews with 82 teachers, carried out as part of a DfE-funded study Evidence-informed teaching: an evaluation of progress in England (Coldwell et al, 2017). It is designed to help teachers evaluate and consider their levels of interaction with evidence in terms of awareness, engagement and use; and illustrate, based on real examples and quotes, what different levels of interaction look like in more or less evidence-informed schools. It is jointly published with The Chartered College of Teachers and is freely available for download on its website.

Designs combining different types of data are increasingly used in educational evaluation, to provide both evidence of impact and an explanation of the processes by which impacts are created. Logic models are visual representations of how an intervention leads via a set of steps from resources and inputs to outputs and then sets of outcomes. Their use has become widespread to underpin evaluations; and they have become of more interest in education as they have been promoted by policy makers and funders including the Education Endowment Foundation (EEF) in England. This paper addresses the question: how can logic models be used to frame and implement educational evaluations using combinations of methods? To do so, the paper draws on theory-based evaluation literature to identify a set of issues to be considered: the role of implementation logic; causal mechanisms; the context of interventions; and the importance of considering and addressing issues around complexity. Using detailed examples from two study designs for EEF evaluations, the paper presents an evidence-informed logic model approach to deal with these issues. The paper concludes by reflecting on the practical and theoretical implications of this approach, laying out a set of key issues to address in future evaluations for which a design framed by an evidence-informed logic model may be appropriate.

Increasingly, policymakers seek to improve the quality of teaching through curriculum innovations and continuing professional development (CPD) programmes. However, engagement by schools and teachers varies owing to mediating influences of neoliberal policies. In this article, we contribute to understanding how these tendencies affect participation. Problematising the notion of context, we examine ways in which systemic influences interacted with participation in a government‐funded mathematics professional and curriculum development programme and also with participants’ purposes.A three‐level clustered Randomised Controlled Trial (RCT) and an implementation and process evaluation were augmented by in‐depth case studies, cross‐case analysis and the application of theoretical constructs to interpret findings. Theories of capital, figured worlds and systemic coupling are utilised to theorise the context.Different levels of engagement are partly explainable by: the interaction of schools’ relative systemic advantage and disadvantage; their orientation and coupling to performativity regimes; and the alignment or dissonance between continuing professional development or change programmes and the pedagogical and CPD cultures and purposes of the ‘actors’ (schools, departments and teachers). Performativity concerns restricted what were considered legitimate outcomes in some case study schools. This depended on teachers and schools’ positioning in terms of relative degrees of systemic privilege or disadvantage—understood as economic, cultural, social and symbolic capital—and also in terms of figured worlds and system coupling. The case studies provide insights into how collaborative professional learning can be fostered more productively. Methodologically, we demonstrate the power of combining methodologies and applying explanatory social theory to augment quasi‐experimental paradigms.

This article considers the role of context in ‘theory-based’ evaluations, particularly those that use chain-type path or logic models. Reflecting on the use of causal models in the school professional development field, a set of underlying features of context is developed: the article proposes that context can be dynamic, agentic, relational, historically located, immanent and complex. These features are applied to a consideration of a commonly observed contextual factor: senior leader support for an intervention. The article argues that actively considering these underlying features can allow for a more sophisticated approach to context and concludes with a set of related interrogatory questions for evaluators, aiming to improve learning in future evaluation.

ABSTRACTThis paper addresses the question: How do teachers and school leaders respond to high stakes testing of pupils transitioning from primary to secondary school? It explores how a new test, the Level 6 test, operated with regard to primary/secondary school relationships in England. It draws on an analysis of qualitative interviews with teachers and school leaders in 20 primary schools that took part in the test, 40 school leaders that chose not to and 20 secondary‐school leaders. Theoretical work on social boundaries is utilised to develop an argument that this test and its results acted as a boundary signifier, crystallising many of the tensions between primary and secondary schools. These tensions included the role of accountability regimes in requiring schools to demonstrate progress; narrowing of the curriculum and teaching to the test; and the extent to which test results can provide a true representation of pupil attainment. We conclude by suggesting the potential of the boundary signifier concept in relation to other tests at the primary/secondary boundary and other key transition points in education, and consider whether such tests can act as an ideal boundary object, serving to help overcome, rather than cement, barriers between schools.

Process evaluation of four strands of the maths and physics supply package, with recommendations for evaluation of the full programme.

One approach to designing, researching or evaluating professional learning experiences is to use models of learning processes. Here we analyse and critique five significant contemporary analytical models: three variations on path models, proposed by Guskey, by Desimone and by Clarke and Hollingsworth; a model using a systemic conceptualisation of learning by Opfer and Pedder; and a cognitive learning model by Evans. To do this, we develop and illustrate an analytical framework focused on model components, purposes, scope, explicit and implicit theories of learning and change processes, agency and philosophical underpinnings. We identify similarities, differences, inconsistencies and limitations in the models. This provides the basis for reconceptualising models as tools to be deployed alongside other relevant constructs and thus the analytical framework can support a more informed selection of theoretical models by researchers and practitioners.

This paper develops a path model of professional development (PD) to explore the relationship between teacher PD and teacher careers and retention. A focussed literature review frames the study, which is based on a survey of 500 + teachers and interviews with 25 teachers. The findings provide evidence of the influence of PD on intermediate outcomes within the model and some evidence of influence on career progression. The paper concludes that a path model can provide a useful frame for examining the influence of PD on career, but the approach is limited given the complex, situated nature of teachers' careers. (C) 2016 Elsevier Ltd. All rights reserved.

Despite the very large literature on teacher careers from an individual perspective, there is relatively little that links the perspectives of teachers themselves to how schools as organisations approach careers. The aim of this paper is, first, to outline how teachers' orientations towards careers change across three dimensions, and, second, to examine how schools as organisations deal with career, developing a model of organisational responses, including developing a concept of career culture', derived from an analysis of interviews regarding the first three years of teaching conducted with senior leaders and second-year teachers themselves. By considering the fit between individuals' career orientations and school career cultures, the paper surfaces both the fluid nature of these orientations and the subsequent potential instability of the fit.