Theory of change (ToC) models articulate how and why implementation interventions are expected to produce specific outcomes and can guide both program design and evaluation. As part of a theory-driven evaluation for a federal initiative to improve non-surgical management of urinary incontinence among women in primary care, we co-developed ToC models for five funded projects. Each model depicted intervention approaches, hypothesized mechanisms of change, and expected outcomes. Using a six-step process which included document review, consensus-based element identification, open coding, iterative refinement, and stakeholder validation, we created and refined a separate ToC for each grantee. The Technical Expert Panel, convened for the broader initiative, provided multidisciplinary feedback to ensure methodological rigor and contextual accuracy. Sharing the models in annual meetings supported cross-grantee learning and revealed common implementation themes: reliance on technology-based patient education and clinician training, challenges with practice recruitment, tailored facilitation to reduce clinician burden, and gaps in patient-level evaluation measures. The ToC models served four primary functions: orientation for evaluators, technical assistance for grantees, monitoring and evaluation of implementation processes, and cross-site collaboration. Key lessons included the importance of early stakeholder engagement, flexibility in model development, and the value of visual tools for communication and shared understanding. Although resource requirements and retrospective development were challenges, ToC models proved to be adaptable, dynamic tools that enhance evaluation and collaboration in complex, multi-site health initiatives. Their systematic application can strengthen future national and international implementation efforts.
Artificial intelligence (AI) is reshaping the conceptual, methodological, and ethical foundations of educational evaluation and assessment. Contemporary assessment models are increasingly challenged by the erosion of clear boundaries between human and machine authorship, complicating assumptions about what constitutes learner-generated performance, knowledge, and competence. The classical assessment paradigm, grounded in the assumption that learner performance reflects individual and independently produced cognition, is increasingly inadequate in educational environments in which AI authors tasks, scores responses, generates feedback, converses with learners, and accompanies students as a partner in the learning processes. This editorial introduces five contributions to this special issue and uses them to advance a broader conceptual agenda for evaluation and assessment in the age of AI. We position the role of assessment in the age of AI across two interconnected areas: assessment and evaluation for AI-integrated learning, where learners collaborate with AI systems to co-construct knowledge and improve performance, and AI-assisted and AI-mediated assessment, in which AI supports, shapes, and increasingly participates in the processes of evaluation, feedback, and decision-making. Across both domains, we develop an intersection of the evidence generated by the five articles included in this special issue, extending beyond the individual contributions to identify broader conceptual, methodological, and practical implications for assessment in the age of AI. We argue that AI’s integration into the learning-assessment ecosystem requires a fundamental reconceptualisation of assessment theory, validity, and learner agency under conditions of hybrid intelligence. We conclude by outlining a future research agenda for evaluation and assessment in AI-mediated educational environments.
As the 2030 Sustainable Development Agenda nears, research efforts have focused on evaluating sustainable development from a macro perspective. However, this macroscale assessment might be influenced by regional variations. Additionally, previous studies have tended to overemphasize quantitative evaluations. To address these shortcomings, this study applies regional sustainable development theory to structure an assessment framework by proposing a hybrid method that combines qualitative and quantitative assessments. The proposed hybrid method includes K-nearest neighbors, hierarchical clustering, a reliability test, and a decision-making trial and evaluation laboratory. It makes three significant contributions: (1) The complex causal interrelationships that are revealed strengthen the theoretical foundation and enhance the understanding of regional sustainable development; (2) the proposed hybrid method considers quantitative and qualitative factors simultaneously, resulting in a comprehensive evaluation; and (3) the simple visual assessment offers clear guidance for resource allocation adjustments. The assessment results reveal that the government prioritizes eco-coordination over the opinions of experts on cultural-environmental livability. On this basis, compared with the 292.64% variation in infrastructure support across regions, eco-innovation exhibits the least variation of 50.00%. The results also suggest that the provinces in Central China have a higher level of consensus in terms of pursuing regional sustainable development.
This study examines job mismatch among immigrant and Canadian-born graduates of Canadian postsecondary institutions. Job mismatch is an important dimension of labour market integration, but widely used measures, such as overeducation based on the formal educational requirements of a job, may not capture the nuance of graduates' employment outcomes. Overeducation and overqualification capture different aspects of mismatch, reflecting the alignment of formal educational requirements and broader assessments of skills, education, and experience. Examining both measures is therefore important for understanding how immigrants and Canadian-born graduates experience the transition from postsecondary education to employment. The analyses utilized data from the 2018 National Graduates Survey to examine 18,794 graduates who completed postsecondary education in Canada in 2015 and entered the labour market by 2018. Bivariate probit models jointly estimated overeducation and overqualification, adjusting for immigration status, sociodemographic characteristics, and educational factors. Mediation analyses assessed whether educational characteristics explained differences between immigrant and Canadian-born graduates. Overall, respondent groups differed in the probability of experiencing distinct combinations of mismatch. For example, joint probability estimates demonstrated recent immigrants were more likely than Canadian-born graduates to report both forms of mismatch simultaneously, whereas established immigrants were more likely to report overeducation in the absence of overqualification. Overeducation and overqualification capture distinct dimensions of job mismatch and offer valuable insights into immigrant labour market integration. Jointly examining both measures highlights the importance of incorporating multiple measures of job mismatch when examining labour market integration and demonstrates the limitations of relying on overeducation alone.
Document-level relation extraction must identify relations when entity mentions and supporting evidence are distributed across multiple sentences. Existing methods often use entity representations that remain fixed after encoding and provide limited support for progressively retrieving and combining dispersed evidence. This study improves complex cross-sentence relation reasoning by allowing entity representations to evolve during inference and by retrieving evidence from both entities in a target pair. The proposed multilayer iterative framework combines the Dynamic Entity Representation Mechanism (DERM) and Dual-Perspective Query Mechanism (DPQM). DERM propagates and updates entity states across reasoning layers using document context. DPQM retrieves evidence separately from head- and tail-entity perspectives and adaptively fuses the resulting representations. The framework was evaluated on Re-DocRED using BERT-base with Adaptive Thresholding and Localized cOntext Pooling (ATLOP) and DocuNet as backbones. Evaluations included overall F1, Ign F1, inter-sentence F1, evidence-based recall, ablation studies, and computational cost analysis. With ATLOP, the proposed method achieved 76.42% F1 and 75.22% Ign F1, exceeding the baseline by 2.72 and 2.46 percentage points, respectively. Inter-sentence F1 increased by 3.74 percentage points. For relations supported by three or more evidence sentences, recall improved from 54.65% to 66.40%. In ablation results, DERM and DPQM provided complementary improvements, with DERM making the larger contribution in complex cases. Explicit entity-state propagation and evidence retrieval from both entity perspectives improved dispersed evidence integration in long documents. The largest gains occurred in cross-sentence relations and instances requiring multiple supporting evidence sentences; however, stronger evidence constraints and calibration are required to reduce unsupported predictions.
