Writing is an iterative process, yet traditional assessments often prioritize the final product over the transformations that shape it. This study investigates the complex nature of revision in student writing through computational modeling, leveraging keystroke data to capture and analyze revision behaviors. Focusing on a dataset of 1,975 annotated revision events from 10th-grade French students, we assess multiple methodological approaches, including rule-based heuristics, machine learning classifiers, and large language models (LLMs). While previous research has demonstrated the feasibility of automated revision detection, we extend these efforts by introducing a novel framework for identifying embedded revisions, i.e. instances where a revision occurs within another. By comparing the efficacy of different computational strategies, our findings reveal key insights into how revisions unfold in realtime writing. Annotations evaluated by agreement measures underline the complexity of the task. This work not only enhances the precision of automated revision classification but also lays the groundwork for intelligent writing support systems that provide targeted feedback to students, fostering a deeper engagement with the revision process.
Adaptive learning technologies observe learner performance, infer mastery, and dynamically tailor instruction. However, this approach can fall short when encountering a learning phenomenon we term “deceptive overgeneralization”, where learners perform correct actions based on incomplete understanding. This phenomenon “deceives” adaptive systems into prematurely stopping necessary practice, leaving overgeneralization unaddressed. To address this, we empirically investigated the mechanisms, risks, and remediation strategies for deceptive overgeneralization through experiments using Intelligent Tutoring Systems (ITSs) for Riichi Mahjong. Experiment 1 provided evidence supporting the effectiveness of ITSs in teaching Riichi Mahjong, showing a large effect size. However, despite their overall effectiveness, Experiment 2 revealed that ITSs may fall short in cases of deceptive overgeneralization, as learners, after demonstrating seemingly satisfactory performance during initial practice, subsequently misapplied learned actions “confidently” in scenarios that did not warrant them. Experiments 3–4 replicated this finding and further revealed that adaptive learning systems relying on observed correctness can prematurely cease practice, leaving deceptive overgeneralization unaddressed. Experiments 5–6 replicated the findings of Experiments 3–4, and also extended the methodology by incorporating tailored practice designed via our systematic detection/remediation procedure, which successfully addressed deceptive overgeneralization. This study contributes both theoretically and practically to enhancing the precision of adaptive learning, enabling more accurate mastery assessments and improved learning outcomes.
Artificial Intelligence Educational Tools (AIED) are poised to transform educational practices by enabling personalised learning, automating administrative tasks, and delivering real-time feedback. However, the ethical and constructive implementation of AIED in K-12 education remains a pressing concern, particularly among teachers. Even when teachers view AIED as a highly trans formative tool, they often remain reluctant to experiment with it in the classroom, primarily because they feel unprepared and lack confidence in using AIED effectively. This study aims to address these gaps by exploring the constructive applications of AIED to empower teachers in enhancing students' cognitive engagement (based on ICAP framework) and higher-order thinking skills (HOTS). A five -month teacher training course was conducted, incorporating pre- and postsurveys to assess teachers' confidence, knowledge, and appropriation of AIED. Teachers engaged in training on AI-related risks and tools, designed lessons that integrated AIED, and gathered student feedback. Teacher training surveys (N = 17) were analysed with paired samples T-tests; deductive content analysis focusing on HOTS and the role of AIED was conducted based on ten lesson plans. For triangulation, descriptive analysis was applied to the student feedback survey (N = 240) about their perception of such learning activities. Overall, the study shows that teacher training empowered teachers in using AIED; the alignment of lesson activities with ICAP principles might depend on the learning community; and AI-supported learning can foster HOTS depending on the type of learning activities and the AIED role.
With the increasing presence of technology in education, foundational skills that can encourage learners' understanding of and meaningful interactions with technology should not be overlooked. Computational thinking (CT) is one such skill with applications to multiple domains. While CT assessment has traditionally focused on final outcomes, a more comprehensive perspective can emerge by also examining the process through means such as log data. The aims of this scoping review are to map how existing research has utilized log data to assess CT and to present research gaps for future studies in this area. Multiple databases from different disciplines were searched for peer-reviewed journal articles and conference papers. Relevant studies analyzed log data of individual learners from open-ended tasks to gauge CT performance. The following five areas were examined from each study: 1) form of CT, 2) education level, 3) log data process features, 4) interpretations of the features, and 5) theoretical contextualization of the features. The analysis revealed text-based programming as the most researched form of CT, followed by block-based programming Additionally, higher education was the most studied level of education. Regarding log data features, attempts and time on task were among the most extracted. In studies justifying the choice of features, a variety of theories and empirical studies were referenced. The suggested interpretation of the features also varied, leading to inconsistent conclusions about what could be inferred from the log data. This valuable insight into existing research patterns can steer future studies towards theory -based measures in underrepresented areas of research, advancing the discussion on how technology can enhance learning. Furthermore, using log data for assessment can enable scalable personalized feedback and scaffolding. Combining a strong theoretical understanding of these topics from educational sciences with the technology-oriented approach to log data could substantially promote new ways of developing effective interventions.
Generative AI (genAI) tools are increasingly being integrated into instructional design workflows for content creation, assessment development, and lesson planning. For novice designers, it is critical to understand whether this integration supports the design process without compromising underlying pedagogical learning. This study addresses this gap through an ecologically valid field experiment within a 14-week graduate course training novice instructional designers. Using a counterbalanced A/B design embedded in authentic coursework, students created eight microlessons, alternating genAI assistance with independent work. Learning of pedagogical principles was assessed via module pre/post-tests and self-efficacy, in teaching practices and genAI use, was measured via course-level pre/post-surveys. Results showed no evidence that genAI use hindered learning, as post-test scores on module content remained stable or improved, despite variations in test form difficulty. Students demonstrated substantial and statistically significant gains by over 10
While learning analytics dashboards have been researched for more than a decade, their adoption in everyday educational practice remains limited, often due to insufficient actionable insights or a lack of alignment with teachers workflows. This paper introduces the Competency Dashboard, a learning analytics dashboard designed to provide teachers with actionable insights into students progress and performance. Developed through an iterative, user-centred design process, the dashboard offers intuitive and accessible visualisations of learning data. Multiple evaluations indicate that it delivers valuable insights and actionable feedback, enabling teachers to understand student performance better and make informed instructional decisions.
Against the widespread use of virtual learning environments (VLEs) in French schools, this research examines their adoption by primary and secondary school teachers by detecting digital maturity profiles based on activity traces. The aim is to analyze the diversity and intensity of the activities carried out in VLEs to identify these profiles and better understand the conditions of their emergence. Based on the statistical processing of VLE data from 144,900 teachers in three academies over 2 school years, we constructed three indicators (MatU, MatD, MatF) to qualify the use in terms of combination, diversity, and frequency of VLE services used. The results reveal five general profiles that differ according to the academic district, the level of education, or the period in which the VLE was deployed. This shows that digital maturity is strongly linked to contextual variables. These profiles suggest that some contexts favor the adoption of complex, collaborative uses, especially in schools that adopted VLE just after the health crisis. The study thus highlights the role of the institutional framework in structuring and sustaining the use of digital technologies in education.
Confusion plays a central role in entrepreneurial learning, as venture creation often unfolds under dynamic and uncertain conditions. Disruptions such as crises or conflicts can lead to dysfunctional routines, triggering cognitive dissonance or emotional distress. These experiences may stimulate deep learning processes, encouraging entrepreneurs to reassess existing mental models and routines and develop new value creation strategies. However, not all individuals possess the same cognitive and emotional capacities to effectively process such events. While prior research emphasizes the transformative potential of confusion, empirical investigations have largely relied on self-reports, which may be limited due to biases, memory constraints, or the implicit nature of cognitive and affective responses. To address this methodological gap, this study introduces an innovative approach to research on entrepreneurial learning by using electrodermal activity (EDA) to measure physiological arousal during confusing events in venture creation contexts. Since skin conductance reflects stress-related changes in sweat gland activity, EDA provides real-time insights into how entrepreneurs react to uncertainty and disruption. By capturing these immediate physiological responses, the study aims to improve the understanding of confusion as both a stressor and a catalyst in entrepreneurial learning and venture development. This contributes to a more nuanced view of how individuals experience and adapt to discontinuities, highlighting the need for objective, process-oriented measurement tools in entrepreneurship research.
Parent and caregiver involvement in homework can be critical to student success, yet many families face barriers such as limited time and content knowledge. We examine two approaches to involving caregivers in homework that students complete with tutoring systems for middle school mathematics. The first, an intelligent caregiver support module, provided real-time, step-level tutoring guidance and SMS notifications to caregivers. The second, a goal-setting contract based on prior non-digital homework interventions, facilitated structured agreements between caregivers and students to set homework goals and rewards. We conducted two four-week studies in American middle schools with diverse student populations, totaling 75 students, to determine which approach engages more households. After low initial adoption of the tutoring support module, the goal-setting contract was introduced at the second study site. While only 12% of the families enrolled in the tutoring feature, 55% engaged with the goal-setting contract when given the opportunity, a significant increase. Short interviews with students and teachers further emphasized the role of household participation-extending beyond parent-child dyads-as siblings frequently assumed homework support roles, contributing to higher engagement with goal-setting. We contribute novel evidence that caregivers were more likely to engage with tools enhancing motivational roles than with real-time instructional guidance in technology-supported homework. Specifically, goal setting emerges as a low-friction strategy to enhance student engagement while mitigating barriers that limit direct instructional support.
Generative AI is increasingly used to enhance educational support in higher education, particularly for distance learning. This paper introduces the Term Paper Recommendation System (TPRS), an AI-powered scaffolding tool designed to assist students in selecting and refining research topics for academic writing. TPRS integrates generative language models with knowledge-based and expert-driven recommendation strategies, dynamically adapting feedback based on the student’s confidence level. The system leverages structured validation, multi-shot prompting with historical supervision data, and semantic similarity for literature recommendations. Deployed within a Bachelor of Arts program in Culture and Social Sciences at FernUniversität in Hagen, TPRS was evaluated using the CRS-Que framework and expert grading of student submissions. Results show statistically significant improvements in topic formulation quality, enhanced engagement, and reduced instructor workload. However, user feedback highlighted the need for improved transparency and control. TPRS offers a novel hybrid architecture that positions AI as a learning scaffold rather than an automation tool. This work contributes to responsible AI integration in higher education by demonstrating how generative systems can support inquiry-based learning while preserving student agency.
Large Language Models (LLMs) have demonstrated potential for (semi-)automating the qualitative analysis of unstructured data, particularly in deductive qualitative coding using codebooks. While prior research has shown the feasibility of this technology, model performance seems to vary depending on the nature of the constructs being coded. However, existing approaches typically apply a single LLM prompting strategy across entire datasets, often of discourse transcripts or questionnaire data, using a single coding scheme or theoretical frame. This paper introduces an adaptive prompting approach where LLM prompts are customized based on researcher-defined or data-driven rules for specific codes. We apply this approach to analyze the description of 35 multimedia learning designs (comprising 758 items/activities) created by teachers in an inquiry-based learning digital platform. Two human coders and an open-weights LLM (Llama 3.3) coded the dataset attending to three different pedagogical frameworks. Our results indicate that datadriven adaptive prompting outperforms uniform prompting approaches (namely, zero-shot, few-shot with/without context) in terms of agreement with human coders and cost. While the improvement is not statistically significant, the approach offers potential advantages for large datasets (especially, considering the costs), highlighting opportunities for human-AI collaboration in educational data analysis.
The research, design, and development of Learning Analytics (LA) and Artificial Intelligence in Education (AIED) technologies are inherently interdisciplinary, spanning fields such as computer science, education, human-computer interaction, psychology, and the learning sciences. Effectively navigating this complex landscape requires methodological approaches that facilitate meaningful collaboration among diverse stakeholders. Co-design has emerged as a promising strategy for aligning LA and AIED technological innovations with the real pedagogical needs of educational practitioners. However, the adoption of LA and AIED technologies in practice is often delayed, frequently due to educators reluctance to alter established pedagogical routines and limited data and AI literacy. To address this challenge, research in teacher education highlights the value of framing co-design not only as a strategy for technology design and development but also as a form of educator learning. This paper advocates for adopting a similar approach within LA and AIED research communities: a shift from primarily viewing co-design as a unidirectional process aimed at eliciting educators input to embracing it as a reciprocal approach that simultaneously supports educators professional growth. By reconceptualizing co-design as a dual-purpose process, LA and AIED researchers can better foster educators agency, AI literacy, and critical pedagogical reflection, thereby potentially enhancing their readiness to integrate innovative technologies into their practice and increasing the long-term sustainability and impact of LA and AIED tools.
While Computational Thinking (CT) has long been a foundational concept in computer science education, the increasing prominence of AI in modern curricula requires exploring if and how AI concepts can be incorporated into CT-based curricula. This study systematically examines the relationship between AI concepts and CT principles by analyzing five widely recognized AI educational frameworks. Key AI concepts are identified, categorized into thematic groups, and if possible mapped to CT elements. The findings reveal areas of convergence where AI concepts align with CT principles and areas where AI provides new dimensions to CT-based curricula. Based on this mapping, we present a case study analyzing a CT-based primary school informatics curriculum and showcase how it can be expanded to integrate AI concepts. The results provide insights into how educators can integrate AI into existing curricula traditionally rooted in classical CT, addressing both the technical and ethical dimensions of AI education. This paper contributes to the ongoing discourse on how AI can enhance and expand the scope of CT, offering a practical approach to the gradual integration of AI concepts within existing educational paradigms.
Research on Technology-enhanced Learning (TEL) has mostly considered, how certain pedagogical functions can be enhanced or supported by technological means. As technologies increasingly become inherent parts of learning and teaching practices, this functional perspective towards the role of technologies in learning is to some extent limited because it underestimates the role technologies can have in the transformation of whole work and learning practices from a systemic perspective. Through the lens of recent models of hybrid intelligent systems, we explore the impact that AI will have on the teaching profession. Through the analysis of six cases that cover a broad range of applications of AI in teaching across a set of educational contexts, we show how AI will replace, complement and augment teaching tasks and thereby affect professional knowledge, skills and attitudes of teachers. Importantly, in a hybrid intelligent system, interactions between actors (human and artificial) will be bi-directional. We propose a framework on teacher-AI complementarity that can guide future research in this area. We also identify some of the important conditions for complementarity, e.g. on how technology is designed, developed and deployed, and how teachers are prepared.
Automated Writing Evaluation (AWE) systems have supported students learning of science writing and teachers evaluation of the writing product. Previous research on AWE has paid limited attention to teachers agency in implementing these tools. As key stakeholders, teachers play an important role in the integration of AWE into traditional classrooms. The present study explores teachers perceptions of an AWE tool, PyrEval, and their experiences implementing it in their science classroom. We conducted a focus group with five middle-school science teachers who co-designed PyrEval with university researchers. Our thematic analysis reveals teachers perceived benefits and limitations of using the AWE tool in their classrooms. Our findings show that teachers perceive the AWE-integrated classrooms as a whole, and adapt their instructional strategies to actualize perceived benefits and minimize perceived constraints. Our study highlights teachers agency in using AWE and has implications to the design of instructional support tools.
Rapidly changing job markets highlight the growing need for continuous and targeted skill development aligned with individual career goals. Existing course recommender systems in higher education predominantly rely on collaborative filtering or content-based techniques, and these systems struggle to match online courses with the targeted job skill demands due to their reliance on historical behavior data, resulting in issues such as the cold-start problem and limited adaptability to evolving career goals. To address these limitations, we introduce CRAFT as a course Recommender System (RS) for the German career context that integrates educational ontologies as a comprehensive domain knowledge and Large Language Models (LLMs) to provide personalized, career-oriented course recommendations along with clear explanations. In this way, CRAFT helps users continuously enhance their skills and make informed learning decisions. Experimental evaluations demonstrate CRAFT's effectiveness in analyzing the skill gap and providing relevant recommendations supported by detailed explanations, increasing user satisfaction. This research contributes to personalized learning frameworks in education by providing personalized, continuous learning opportunities through effectively linking online courses with evolving occupational competencies.
As concept maps gain prominence in education, ensuring reliable and efficient assessment methods becomes increasingly important. However, automating this process remains a complex problem. Due to their free-form nature, assessing a concept map is subjective, as there is no single "correct" way to design one for a given topic. To address these challenges, this systematic literature review identifies techniques and criteria currently employed to automate the evaluation of concept maps. In addition to conventional systematic review methods, the study utilized DeepSeek as an auxiliary tool for paper selection, validating its results against selections made by two independent researchers via the Kappa test. The findings highlight recent advancements in evaluation techniques for using AI in qualitative assessment. Still, gaps persist in assessing collaborative and multilevel educational contexts. Additionally, the field lacks standardized evaluation criteria, including established datasets or universally accepted metrics for comparing the performance of different automated assessment approaches. By synthesizing current practices in automated concept map evaluation, this study aims to foster discussions on standardizing evaluation metrics and to provide a comprehensive overview of the state of the art in this evolving domain.
Group awareness tools (GAT), or widgets, subtly scaffold computer-supported collaborative learning (CSCL) by visualizing qualities of the learning process supporting learners' self-regulation. Some central interaction qualities, like transactivity, i.e., operating on the reasoning of the other, and epistemics, i.e., how learners are using concepts on the task, have been difficult to analyze (and visualize) in run-time. In a closed online environment, a transactive and epistemic GAT provides real-time analysis of the discourse using the XLM-R multi-lingual masked language model. The discourse analysis system runs on an independent server allowing local control of data and manipulation of feedback visualization for participants within the chat environment. A pilot test of the widgets provided initial findings that suggest effective functioning of this GAT. This demo allows participants to engage in an online discussion in pairs, and experience how the widgets visualize their conversation in terms of how much they build on others' reasoning and what concepts they use to solve the task, which in turn can influence their subsequent discourse patterns.
This design -based research investigates the impact of different debriefing techniques on student learning following Collaborative Learning Flow Patterns (CLFP). Following a literature review, findings were incorporated into the design phase, ensuring alignment with multimedia principles. In five experimental cycles participants engaged in tasks following the Pyramid CLFP, generating 328 analyzable data sets. The study explored various debriefing techniques, including Debriefing Scripts and visualizing student artifacts using Word Clouds, Quotes with Emphasis, Heatmaps, and Concept Maps. Results indicate that structured and visual debriefing methods notably enhance student learning outcome. Word Clouds facilitated broad discussions in open-ended tasks but lacked structural depth. Concept Maps provided clear representations of complex relationships, proving highly effective for structured tasks. Quotes with Emphasis supported student reflection but risked reinforcing misconceptions, while Heatmaps illustrated collective tendencies but lacked relational detail. These findings highlight the importance of carefully designed debriefing techniques in optimizing student learning outcomes and provide the basis for further advancing the implementation of these in pyramid collaborative learning.
Multimodal learning analytics have significantly improved our understanding of learning processes. One of the most discussed advantages of multimodality is the holistic view of the learning processes. In this contribution, we explore whether giving feedback based on multimodal analytics benefits learners. We designed and developed a novel system that collects learners' eye-tracking and heart rate variability data while they debug a given code. The system computes the cognitive load from the eye-tracking data and physiological stress from the heart rate variability data. If the cognitive load and/or physiological stress were more than a threshold, the system would provide feedback on the problem. We conducted a controlled study involving 120 students, with one control condition and three experimental feedback conditions (cognitive load only, stress only, and both cognitive load and stress). The results show that the debugging performance was improved with the feedback. More importantly, we found that the feedback also positively affected students' cognitive load and physiological stress. We discuss the implications of these results in the view of scalability and impact.
