Assessing social impact remains challenging for small-scale, early-stage projects in low- and middle-income countries (LMICs), where existing Social Impact Assessment (SIA) frameworks are often too complex for organizations with limited capacity. This study develops a flexible SIA Application System tailored to innovation-oriented pilot projects operating under high uncertainty in LMICs. Refined through a qualitative case study of the Global Innovation Fund—using semi-structured interviews and document analysis across agriculture, water, and electricity sectors—the system integrates three components: (1) a refined framework of core assessment dimensions; (2) a structured process for local scoping and indicator selection; and (3) adaptive functions aligning assessment with project maturity, from pre-pilot to post-pilot phases. By repositioning SIA as a staged, learning-oriented process, the system balances feasibility and accountability, offering a practical tool for adaptive decision-making in high-uncertainty LMIC pilot contexts.
This article presents a reflexive re-analysis of an externally commissioned real-time evaluation (RTE) of the COVID-19 response in Västmanland County, Sweden. Drawing on material generated during the original evaluation – 33 semi-structured interviews, observations of coordination forums, internal documents, and contemporaneous methodological logs – it examines how learning, authority, and legitimacy were negotiated while crisis governance was still unfolding. The findings show that evaluation under crisis conditions did not operate simply as a neutral diagnostic exercise. Rather, evaluative judgement was shaped by defensive routines, mandate ambiguity, uneven documentation, and legitimacy-sensitive negotiations over wording, evidence, and public representation. At the same time, the evaluation created limited but real openings for reflection, dialogue, and modest coordination adjustments. The article argues that the case illuminates, in concentrated form, more general problems of evaluability, authority, and interpretation in learning-oriented evaluation under institutional strain.
This article discusses how evaluation bricolage, when foregrounded by relationality and informed by multicultural validity, offers potential for transparent decision-making within evaluative bricolage. The authors intentionally explore the risks and the opportunities that lie within weaving paradigms, something often not spoken of in evaluation bricolage. In doing so, they advocate for a shift in paradigm to indigeneity that centres being in relationship and relational accountability as primary conditions of bricolage. The authors state that applying multicultural validity as a supportive decision-making framework can broaden our ability to centre context, culture and values within bricolage. In turn, ways of knowing, being and doing that strengthen the overall trustworthiness of evaluation, including bricolage, can be brought forward. Finally, the authors illustrate, through an example from evaluation practice, how multicultural validity centred on relational accountability acknowledges and legitimises diverse worldviews, value systems and knowledge traditions.
This article explores the tensions that arise in the evaluation of programs for the prevention of violent extremism (PVE), using Bourdieu’s theory of fields and frame analysis as analytical lenses. Based on qualitative data collected in nine Western countries, the study highlights how divergent field logics shape actors’ expectations, interpretations, and practices around evaluation. It identifies four central tensions—particularity vs harmonization, involvement vs independence, transparency vs confidentiality, and program vs practice evaluation—each reflecting structural and ethical dilemmas. Rather than viewing these tensions as obstacles, the article argues that they can become productive through processes of frame alignment, especially when evaluators act as mediators between fields. Although not exclusive to PVE, these challenges are intensified by the political sensitivity and ethical demands of this domain. The article concludes by emphasizing the need for relational competence, field awareness, and inclusive approaches in building meaningful and legitimate evaluations.
It is increasingly recognised that ‘big problems’ in health and social care require well-designed solutions and robust evaluation, including attention to costs as well as outcomes. This is especially important in the pervasive environment of resource scarcity. Decisions need to be made that consider not only whether and how an intervention works but also affordability and efficiency in different circumstances and populations. As researchers tackle these complex issues, interest in bringing together realist and economic evaluation is growing. Responding to this, we discuss how using programme theory within a novel realist economic evaluation approach can be used in articulating how interventions may (or may not) bring about cost-effective change. We identify what initial programme theory in realist economic evaluation might look like in practice, drawing on data from three pilot realist economic evaluation studies.
Public policies increasingly promote interventions grounded in territorial specificities, yet evaluation practices remain poorly equipped to take territory into account. While territorial impact assessments (TIAs) are intended to address this, they rarely do so in practice. We argue that a key reason for this lies in the predominantly quantitative and top-down way in which TIAs conceptualize territory, despite its inherently multidimensional nature. As a result, important territorial assumptions underlying place-based policies risk remaining invisible. In response, we developed MOSA, a multidimensional analytical framework, informed by regional studies literature, and tested this with four French non-profit organizations involved in TIAs. We used two methods: a monographic analysis of documents and qualitative data collection through observations and semi-structured interviews with evaluators. We found that by making territorial assumptions explicit, MOSA can help prevent misalignment between evaluation approaches and territorial foundations, while offering the possibility of developing more complexity-sensitive evaluation practices.
Organizations delivering developmental services for people with disabilities increasingly face demands to demonstrate their impact. To capture the complexity, personalization, and contextual variability of these services, the Human Capabilities approach is a promising perspective since it shifts the focus from service provision to what individuals are effectively able to do and be, also considering both personal agency and structural constraints. This study proposes an outcome-based framework aligned with the Human Capability approach and developed by integrating the Theory of Change and Realist Evaluation. The findings draw on thematic analysis of 31 interviews with staff from seven Training Services for Autonomy in Italy. The resulting framework maps resources, activities, and outcomes in a non-linear structure, accommodating the diversity of user needs and trajectories, and includes a dual-level performance system focused on individualized assessment of user capabilities and organizational performance. A core feature of the model lies in its integration of user-centered goals with contextual and mechanism-sensitive evaluation, ensuring both strategic orientation and adaptive learning in complex human service contexts.
The context in which social interventions are piloted and evaluated is critical to success, and there is room for further theoretical and empirical work. Limited knowledge exists about ‘creating the conditions’ for interventions to succeed and about dealing with disruptive changes in context. This article presents a ‘theory of disruption’, to go alongside theories of change and harm. After discussing the challenge of context, and gaps in research, we present a worked example that uses a reoriented logic model to describe the impact of unexpected changes in the wider context of a recent evaluation. This considers the COVID-19 pandemic as a disruptive force on an ongoing school-based intervention. The pandemic is considered an intervention in itself, and the disruptive effects it created are mapped and discussed. The article concludes by considering other uses of the method, including to theorise and measure the effects of a wide range of changes in context.
This invited commentary, given at an institute of public health research on a medical school campus, begins by distinguishing evaluation from assessment to ensure that readers share my understanding of the object of analysis. Having set that stage, I submit some observations about the conventional frame for evaluating public programs and policies. I then offer some thoughts on where I believe the field is heading, or perhaps ought to be heading. Much of the substance of this paper draws from a book that my colleague, Emily Gates, and I have prepared on the roles of values, valuing, and evaluating in social research.
This “roundup” review of evaluation blogs, podcasts, and webinars covers the second half of 2025. It discusses the rise of two new podcasts on realist evaluation, reflecting on perspectives of realist evaluation’s distinctiveness and the state of current debates on contentious issues such as on realist interviews and realist trials. It also continues to trace the debate regarding what counts as evidence discussed in roundup review IV in blogs regarding discussions at the Australian Evaluation Society Conference. The review identifies evidentiary criteria as a possible entry point to move beyond debates on evidence hierarchies. It highlights reflections on the benefits and challenges of pre-defining criteria for making evaluative judgements related to complex change processes.
This article presents a realist-informed programme-level theory of change developed for a multi-site evaluation of a focused deterrence intervention aimed at reducing serious violence in five UK cities. Focused deterrence, a complex, cross-agency approach, requires theoretical tools that can account for local variation, emergent mechanisms and shifting implementation contexts. Using a five-stage process involving document review, fieldwork, workshops and qualitative interviews, we developed and refined the realist-informed programme-level theory of change to reflect variation in delivery, updated assumptions and context-mechanism-outcome configurations. Our findings reveal divergent delivery models, re-interpretations of core intervention resources and associated mechanisms, non-linear behavioural trajectories and participants' strategic responses to perceived risks and opportunities. The final model offers a transferable framework for understanding and evaluating how complex interventions unfold across systems. We conclude by outlining lessons for evaluators seeking to develop theory-informed, complexity-aware theories of changes in real-world settings. These are particularly relevant in contexts involving cross-sector coordination, multiple delivery systems and flexible but systematic evaluation designs.
There have been many calls for systems-informed approaches to evaluation. Population health interventions, which often focus on improving the upstream determinants of health through large-scale change, seem well-suited to systems-informed approaches to evaluation. The purported benefits and appropriateness of taking a systems-based approach to population health intervention evaluation have been extensively discussed, and there is a growing, but still limited, number of applied case studies that operationalised these calls. Here, we reflect on insights gained from recent experiences across three evaluation case studies of: a late-night levy to support local policing (UK), the tiered soft drinks industry levy (UK) and a value-based tax on sugar-sweetened beverages (Barbados). Building on theoretical work in this area, we illustrate through applied examples how a systems-informed approach can help to cast a wider net for potential impacts, be integrated with an effectiveness perspective to produce deeper insights and support greater engagement with non-linearity.
The realist evaluation approach has become firmly established within the field of evaluation. Reflecting its sustained and increasing uptake across diverse fields, a growing number of reviews have, over the years, examined practical applications of realist evaluations. Drawing on an umbrella review of 23 published reviews of realist evaluations, this article takes stock of key challenges in realist evaluation and proposes practical principles for addressing them. The proposed principles are designed to promote greater methodological congruence, coherence and transparency in the design and implementation of future realist evaluations.
This article investigates how an imbalance of institutional subsystems can influence the quality of the institutionalisation of evaluation. Building on the concept of coercive isomorphism and the recent country-comparative literature, it theorises that structural asymmetry, where the ‘political system’ administers competitive funding structures, forces practitioners in the ‘social system’ to adapt. These ideas are applied to the policy area of preventing and countering violent extremism (P/CVE), where the literature and initial empirical evidence suggests that governments’ funding power can compel practitioners not only to institutionalise evaluation, but also to adopt standardised, output-focused forms of evaluation. This latter form of coercive isomorphism, driven by the political need for quick accountability, severely diminishes the potential for organisational learning and risks reducing evaluation to a symbolic practice. Nevertheless, the article recognises an adaptive capacity for constructive dialogue and calls for future comparative research looking at different policy areas.
Drawing on data from Cohesion Policy funding, evaluation tender notices, and contract expenditures, complemented with semi-structured interviews with commissioners and contractors, the analysis identifies the key factors shaping the demand and supply of market-based evaluation services. Key issues encompass the prevailing large-scale procurement model, which influences industry structure, market dynamics, and evaluation quality. The findings call for active market stewardship to safeguard methodological diversity and strengthen commissioning capacity, rather than a straightforward segmentation strategy. In particular, strengthening public managers’ capacity to formulate focused evaluation questions and prioritize causal analysis could enhance the quality and policy relevance of evaluations informing Cohesion Policy programming.
While impact evaluation has moved away from strictly behavioural approaches (as embodied by randomised controlled trials) to include contexts and mechanisms, the role of institutions as explanatory mechanisms has so far been understudied. This lack of reflection on institutions in impact evaluation contrasts with the centrality of neo-institutionalism in policy analysis. This article draws on the inputs of these theoretical reflections developed in policy analysis to theorise how institutions can be analysed as an explanatory mechanism in impact evaluation. The fruitfulness of this perspective is empirically illustrated by the case of an impact evaluation of French community centres. The article shows how community centres as institutions mediate the impact of a variety of community-based interventions through fostering a welcoming culture, encouraging the expression of residents' needs and wishes, and relying on organisational flexibility and embeddedness.
Understanding the mechanisms and contexts that drive the success of complex health interventions remains a challenge. Realist methods, grounded in scientific realism, generate context-sensitive programme theories to explain how, why and for whom interventions work, but these theories often lack structured operationalisation to inform comparison or intervention design. Behavioural science, by contrast, systematically identifies and modifies behaviour change mechanisms using theory-driven frameworks, but has been criticised for insufficiently considering context. Integrating these may enhance the precision, standardisation and applicability of realist programme theories. This novel approach leverages behavioural science concepts such as behaviour change techniques and mechanisms of action to clarify mechanisms, and uses the idea of behavioural settings to explicate context. Together, this establishes a common language for programme theory formulation, making them more structured, testable and transferable. A five-step framework for integration is proposed for realist studies, facilitating more precise and transferable theories that support intervention design and policy translation.
Moving beyond gold standard thinking in evaluation methodology requires robust, alternative frameworks for methodological choice and justification. We develop such a framework, which we term 'Principled Adequacy for Purpose'. We develop this account by considering recent work centring both the role of questions and values for methodological choice. While we argue that both approaches make important improvements over traditional evidence hierarchies, these frameworks by themselves also face significant limitations. We consider that combining these frameworks, while giving greater consideration to the notion of evaluative purpose, affords better guidance for methodological decision-making in evaluation. For this, we draw on recent work on Adequacy for Purpose in model evaluation, to combine these approaches under the 'Principled Adequacy for Purpose' umbrella.
Foundations play an important role in modern societies, but not much is known about the quality and outcomes of the many diverse activities they support. There is an increasing interest in evaluation policy as a systematic approach to evaluation in foundations. We study the extent to which foundations in Denmark publish an explicit evaluation policy, as well as their requirements on grantees and the evaluative information that foundations offer back to stakeholders and society. Based on a systematic study of the websites of a representative sample of 20 foundations, we find that only three have explicit evaluation policies, and they are not very comprehensive. The majority of foundations do not engage in evaluation, and many do not provide any substantial evaluative information on their website. We offer the extraction/public provision ratio (EPPR) as a conceptual device for further studies. We also discuss barriers and potential improvements in foundations' engagement with evaluation policy.
This article presents an expanded Garden of Evaluation Approaches, a multidimensional framework that bridges theory and practice to strengthen evaluation in complex global contexts. The Garden now maps 13 approaches across intersecting dimensions and introduces two new elements: evaluand focus and lifecycle timing. Developed through systematic review and rubric-based analysis, it operationalises theoretical distinctions into visual profiles and evidence-based resource guides, enabling comparative learning, hybridisation, and context-sensitive design. By integrating underrepresented paradigms alongside established models, the Garden advances pluralism, equity, and methodological innovation. At the same time, it contributes to scholarly debates on classification logic and paradigm integration, offering a dynamic platform for research, pedagogy, and capacity-building. Designed as a living framework, the Garden invites co-creation and adaptation, positioning itself as both a practical decision-support tool and a catalyst for advancing evaluation theory and global practice.
