Evaluation methods have recently expanded, with options ranging from remote sensing to qualitative and quantitative methods, but deciding which to use can be challenging. This paper explores three evaluation methodologies used to retrospectively evaluate an agricultural app in Nepal. This includes using remote sensing to measure vegetation health, a limited quantitative survey to explore app usage and GPS accuracy, and qualitative interviews to contextualize app usage. Remote sensing outcomes show no differences in outcomes for farmers who utilized the app more, while the quantitative survey highlights that app usage was limited and the GPS data should be approached with caution. Qualitative results indicate that while the app attracted interest from local farmers, continued support could have increased usage. While these methods are useful individually, only by laying them together can we evaluate the strengths and weaknesses of each one, while gaining a holistic view of the app rollout and training.
This study addresses a central challenge in contemporary evaluation: how to assess the plausibility of causal reasoning underlying policy interventions in complex governance systems. Using the Dutch function mix reform as an illustrative case, the study reconstructs and evaluates its program theory (PT), which aims to improve teacher retention and instructional quality through differentiated roles, professional recognition, and financial incentives. Program theory reconstruction (PTR) is combined with realist evaluation and argumentation analysis to make evaluative reasoning explicit and to assess the coherence and contextual plausibility of causal assumptions. Findings indicate that the PT is internally coherent but contextually fragile, with key mechanisms operating only under specific enabling conditions, including leadership, human resource management (HRM) capacity, and organizational trust. The study demonstrates how PTR can function as a meta-evaluative framework that enhances transparency in causal reasoning and supports learning-oriented evaluation.
Positionality statements too often take the form of a static list—a demographic snapshot presented once and left behind. This article argues for something else: positionality as a process of becoming. Drawing on ethnographic, feminist, and critical theory literatures, as well as my experiences as an immigrant, multilingual evaluator, I propose transient positionality as a dynamic, relational, and context-dependent framework. It encompasses three interconnected dimensions: fluid positionality, which shifts across time and contexts; relational positionality, which emerges through relationships with interest holders; and perspectival positionality, a term I introduce to describe how positionality is shaped by the social locations of others. Drawing on the concept of the ethnographic toolkit, I explain how this transience is not passive but requires active, intentional engagement. I conclude by considering how transient positionality can enhance the quality of evaluation by making the inquiry process more authentic and transparent.
Designing culturally responsive programs requires evaluation approaches grounded in place, relationships, and community-defined values. This paper presents a case story of a community science engagement program across rural and Tribal communities. Guided by Indigenous evaluation principles, the program team developed the Yucca Logic Model, a place-based metaphor-based logic model grounded in the life cycle of the culturally and ecologically significant yucca plant. The Yucca Logic Model reflects an Indigenous, culturally grounded approach for program planning, implementation, reflection, and learning that centers a cyclical process contrasting the linear, backward design approach of Western logic models. The Yucca Model was evaluated through Ripple Effects Mapping, a participatory evaluation process, by adapting the yucca metaphor to support storytelling, visual mapping, and collective reflection. This approach surfaced intended and emergent outcomes, foregrounded relationships, and documented non-linear change over time. This case story offers a practitioner-oriented example of culturally responsive, metaphor-based evaluation aligned with Indigenous worldviews.
This article examines the significance of positionality and reflexivity in evaluation practice through the reflective case studies of two evaluators. Positionality refers to how evaluators’ social identities and experiences shape their perspectives and methodologies, influencing their interactions within diverse communities. The first case examines how a Mexican American evaluator navigates expectations of representation and cultural brokerage among Latino communities in Los Angeles, uncovering the limitations of shared identity as a basis for culturally responsive practice. The second case, set in South Korea, highlights how evaluators’ internalized biases and epistemological tensions impacted a cross-cultural evaluation, underscoring the complexities of shared cultural identities. Collectively, these examples emphasize the necessity of ongoing critical reflection to acknowledge implicit biases, challenge dominant paradigms, and authentically elevate marginalized voices. The authors advocate for reflexivity as an essential professional practice toward fostering equity, inclusivity, and social justice within evaluation.
