AI increasingly shapes high-stakes decisions in domains such as education and healthcare, underscoring the importance of engaging K–12 students in AI literacy experiences that foreground societal dimensions of AI, including fairness, bias, and accountability. This paper presents a story-driven, game-based learning environment designed to support middle school students’ reasoning about these societal dimensions. In the game, students take on the role of decision-makers evaluating AI models in simulated clinical scenarios, navigating trade-offs between accuracy, training data representativeness, and model appropriateness for specific populations. We report findings from a classroom-based pilot study with adolescents (ages 11–14) examining how students engaged with issues of fairness, bias, and accountability during the intervention. Data sources included brief pre/post knowledge assessments and attitudinal surveys. Results suggest that participation was associated with improvements in students’ understanding of AI-supported decision-making and shifts toward more critical perspectives on AI capabilities and limitations. Given the short duration of the intervention and the modest scope of the assessment measures, these findings should be interpreted as preliminary. Nonetheless, this work provides early evidence that narrative-centered, game-based approaches can support student engagement with the societal dimensions of AI and offers design insights for integrating ethical reasoning into classroom-based AI learning experiences.

IntroductionSupporting meaningful student engagement is an important goal of educators and educational researchers. However, there are many variations in definitions and measures of engagement, making it challenging to recognize and design for deep engagement. To address this challenge, the purpose of this work is to interrogate the ways that different theoretical and methodological choices impact how we describe student engagement. MethodologyWe applied two student engagement coding schemes to classroom video from two computer- supported collaborative learning (CSCL) contexts. We compare how each coding scheme made different types of engagement more or less visible and explore potential decision points and synergies.FindingsThe coding schemes differed in many ways, including their theoretical grounding, level of analysis, and inclusion of various dimensions of engagement. We argue that each of these methodological choices orient researchers to video data differently, which has implications for how researchers make sense of group work, next analytical steps, and claims. There are also substantial challenges that researchers may encounter when using engagement coding schemes outside of the contexts in which they were developed, including achieving reliability and the relevance of indicators of engagement to the novel context.DiscussionThis study contributes to our understanding of how to make sense of engagement in classroom video and the synergies between different coding choices and theoretical perspectives.

Fostering engagement among undergraduate computer science students in large-lecture settings can be challenging for instructors. Didactic teaching styles common in such lectures may not be as effective as dialogic teaching, but the overhead involved with dialogic teaching may preclude its use in large introductory computer science courses. Classroom response systems such as multiple-choice "clicker" systems provide a way for students to engage with an instructor, but evidence suggests that multiple-choice questions may not foster deep thought in the way that open-ended questions do. Open-ended questions foster deeper engagement and AI-enabled learning analytics offer a powerful method of automatically assessing student responses, but grading text responses produced by students and summarizing class-wide performance during lectures presents unique difficulties, especially for algorithmic questions prevalent in computer science lectures. We present a system for delivering, assessing, and summarizing students' responses to open-ended question prompts during such lectures. A central part of this system is the effective summarization of topic-level student responses to provide actionable feedback to the instructor. This poster presents an early phase of this summarization work, analyzing both the quantitative semantic accuracy and qualitative usefulness to instructors of an LLM-based system that summarizes student feedback. Semantic accuracy is found to be high and consistent across different summarization methods. Instructor interviews identified several key qualities of summaries that would be useful, including format preferences and features that would allow selective insight into student responses related to misconceptions.

As AI systems increasingly make autonomous, goal-directed decisions, enabling young learners to understand how these systems reason and act is a central challenge in AI education. However, many introductory AI learning experiences treat AI as a black box, limiting students’ opportunities to reason about how AI systems represent goals, make decisions, and act in the world. This paper presents a narrative-centered game-based learning environment that makes AI planning visible as a form of goal-directed reasoning to support students’ emerging AI literacy. Within the learning environment, students engage in a story-driven investigation, collaborating with an AI-driven agent to solve consequential problems. Students construct and revise representations of goals and actions using a block-based interface, observe how an AI planner generates and executes plans in the game world, and iteratively refine their representations through a Use–Modify–Create scaffolding progression. The narrative context provides concrete stakes for planning decisions, while the behavior of the AI-driven agent serves as interpretable feedback that supports students’ sensemaking about goal-directed AI reasoning. We report findings from an implementation of the game with students ages 10–11, examining learning outcomes using pre- and post-assessments of AI knowledge. Results demonstrate statistically significant gains in students’ understanding of AI planning concepts related to goal representation and decision-making. These findings illustrate how narrative-centered game design can make goal-directed AI reasoning visible and foster young learners’ AI literacy.

Artificial intelligence (AI) is increasingly shaping how young learners interact with digital technologies, yet many upper elementary students engage with AI systems passively and develop intuitive and sometimes inaccurate conceptions of how these systems work. This study examines the Foundational AI construct within a refined learning progression (LP), exploring how scaffolded instruction and dynamic assessment support conceptual shifts in students' understanding of how AI collects, learns from and uses data to make decisions. Drawing on Vygotsky's zone of proximal development and synergistic scaffolding theory, we refined the foundational AI construct of a five-level LP and designed a two-phase activity grounded in this LP to elicit and support student reasoning through structured tasks, informational scaffolds and facilitator prompts. Through mixed methods analysis of clinical interviews with 13 fourth and fifth graders (9-11 years), we identified recurring misconceptions and tracked shifts in student reasoning and movement along the Foundational AI construct of the LP. Furthermore, we examined one student's trajectory in depth to illustrate how dynamic assessment can function as a responsive instructional tool. Findings provide initial empirical insight into how scaffolded LP-aligned instruction, paired with dynamic assessment, can support young learners' movement from surface-level ideas to more structured understandings of how AI systems function. These insights contribute to the design of developmentally appropriate and contextually responsive AI learning experiences for primary education.Practitioner notes What was already known about this topic? Many young learners interact with AI technologies (eg, voice assistants, recommendation systems) but often hold surface-level or inaccurate conceptions of how AI works. AI literacy frameworks exist, but none currently provide scaffolded pathways that align with young students' developmental readiness or explicitly address their initial misconceptions What this paper adds? Provides an initial empirical examination of a refined five-level Foundational AI construct within a broader Learning Progression (LP) for upper elementary students. Demonstrates how LP-aligned scaffolded instruction, using tasks, just-in-time informational supports and decision trees, can guide students from intuitive ideas to more data-centered reasoning. Uses dynamic assessment to track and support conceptual growth, providing insight into students' readiness to reason about AI systems. Implications for practice and/or policy Scaffolded LPs that integrate structured tasks, informational prompts and dialogic facilitation can help support developmentally grounded AI instruction that is responsive to learner needs. Dynamic assessment frameworks can help researchers and educators capture students' shifts in reasoning, differentiating between ideas students can articulate independently and those requiring additional support. Designing layered, responsive scaffolds that actively elicit student reasoning and provide opportunities for reflection can support educators in guiding students' conceptual growth in AI literacy.

Effective classroom teaching requires instructors to be responsive to their students, such as by pivoting their lectures in real-time to address common misconceptions that their students may have developed. Classroom response systems such as multiple-choice "clicker" systems are one method by which instructors can gauge their students’ understanding during classroom lectures, but open-ended questions that prompt students to engage in self-explanation are better suited to promoting critical thinking. Additionally, analyzing students’ natural language responses typically requires time-consuming manual analysis, which makes it challenging to implement in a classroom setting. To address this challenge, we present an LLM-driven method for automatically assessing students' responses and generating an aggregated summary of LLM-based evaluations for their self-explanations during undergraduate classroom lectures. Our approach extracts relevant knowledge components for a given question, tags students’ responses according to whether they correctly address each knowledge component, and generates class-level summaries that highlight common misconceptions and gaps in knowledge to support instructors in pivoting their lectures in real time. We evaluate the system’s effectiveness at these tagging and summarization tasks on data from an undergraduate computer science course, using quantitative and qualitative metrics such as relevance, sufficiency, hallucination rate, and alignment with instructional goals and desired feedback format gathered through instructor interviews. Results suggest that the explanation-based classroom response system can accurately analyze students’ natural language explanations.

In computer-supported collaborative learning environments, analyzing student dialogue is essential for understanding collaborative problem-solving behaviors and supporting effective learning. Prior work often treats all dialogue interactions uniformly, failing to capture how specific dialogue interaction differentially impact learning experiences and outcomes. To address this limitation, we introduce a dialogue-based learning analytics framework that integrates weighted temporal clustering of dialogue with large language model-based interpretation. Our framework identifies student interaction patterns most predictive of group learning gains and uses these insights to enable early prediction of learning outcomes and generate pedagogically meaningful interpretations. We evaluate our framework on collaborative dialogue from middle school students engaged in a collaborative game-based learning environment. Our results show that our framework achieves 83.1% accuracy in learning outcome prediction. In addition, expert evaluations and case studies demonstrate that the identified weighted dialogue patterns reflect key collaborative problem-solving behaviors recognized as important in collaborative learning. By surfacing high-impact interaction patterns and enabling prioritized interpretation generation, our framework provides a promising approach for accurately analyzing students’ collaborative dialogue.

Narrative game-based learning environments (GBLEs) offer learners an exploratory platform for engaging in self-regulated learning (SRL) processes to deepen their understanding of complex STEM topics. However, the affordances of narrative-based GBLEs can allow learners to avoid intended learning materials and processes through (mal)adaptive behaviors, such as gaming the system. The current study examines high school students’ (N = 204) gameplay with Crystal Island, a narrative-based GBLE, and identifies how the proportional time spent gaming the system impacts their learning outcomes. We identified two gaming behaviors that were highly correlated and used principal component analysis (PCA) to combine the behaviors into one principal component (PC) of gaming behaviors. Regression analysis identified a significant negative relationship between PC gaming behaviors and students’ learning gains, indicating that the more time a participant spent gaming the system, the less microbiology content they learned. We discuss future directions for the development of frameworks to distinguish adaptive versus maladaptive behaviors using granular trace data and multimodal data, such as concurrent think-alouds, to gain insight into students’ motivations and intentions behind their behaviors.

In STEM education, game-based learning environments (GBLEs) have become prominent platforms to scaffold students’ self-regulated learning (SRL) strategies. This study challenges the assumption that all engagement in GBLEs is equally beneficial by distinguishing cognitive from behavioral engagement within the Integrative Model of Multidimensional SRL Engagement (IMMSE). We analyzed data from 227 high-school students playing Crystal Island, a narrative-centered microbiology GBLE, to examine how these engagement types during the forethought and performance SRL phases predict learning gains and problem-solving accuracy. Overall, results showed that cognitive engagement significantly predicts learning gains, whereas behavioral engagement does not. This result supports IMMSE’s distinction between surface-level actions and deeper processing, indicating that meaningful learning depends on strategic cognitive engagement. We discuss implications for designing adaptive scaffolds that dynamically balance engagement dimensions across early SRL phases to optimize both efficiency and effectiveness.

Narrative-driven game-based learning environments (GBLEs) are characterized by their capacity to offer learners autonomy in navigating immersive virtual spaces, facilitating self-directed or independent exploration, and interactive learning experiences with complex content. Such narrative-focused GBLEs could thus provide dynamic interaction with educational content in ways traditional instructional approaches cannot replicate at scale. Prior research shows that restricted navigational movement and in-game interactions in GBLEs have improved learning outcomes. Additionally, prior research done on immersive environments has shown that in-game affordances promote learners’ sense of presence, a fundamental psychological experience of immersion that can lead to significant learning outcomes. However, there is limited empirical understanding of how learners’ initial navigation strategy choices unfold in unrestricted GBLEs. It also remains unclear how these approaches differ based on individual differences such as prior gaming experience and perceived presence. This study identifies two ‘navigational strategies’ used by high school learners (N = 152) during gameplay with a narrative-driven GBLE, Crystal Island, and examines how their prior video game experiences and self-reported sense of presence influenced their learning outcomes. Our results showed that learners with higher video game experience engaged significantly more in independent exploration than those with less video game experience. In contrast, we found that learners who followed pedagogical scaffolding and structured guidance to initiate navigation reported a greater sense of presence during gameplay than those who independently explored the environment. Collectively, these results highlight the importance of diverse individual differences in informing the design of adaptive scaffolding and navigation support within GBLE.

Active learning in introductory programming depends on frequent, high-quality feedback, yet instructors often struggle to deliver consistent support at scale. In this poster, we introduce INSIGHT, an AI-driven classroom assistant designed to promote active learning in introductory programming (CS1) courses through scalable, personalized, and explainable feedback. The assistant combines the generative capabilities of large language models (LLMs) with instructor-in-the-loop authoring and an explainable code analysis engine to ensure pedagogically aligned support. Instructors can co-design problems with LLM assistance, provide exemplar solutions, define common student errors, and author targeted feedback. The AI engine analyzes student code submissions, identifies mis-conceptions, and maps them to instructor-verified feedback in real time. INSIGHT is designed to ensure that key educational concepts and common misconceptions are explicitly addressed by instructors, while also leveraging LLMs to provide reasonable feedback for novel or edge-case solutions that instructors may not have anticipated. By combining instructor expertise with the flexibility of generative AI, the assistant helps close feedback gaps and ensures more comprehensive coverage of student learning needs, especially in large or diverse classrooms with limited instructional support.

This study develops and validates a human-centered approach to measuring small-group, multidimensional engagement in a game-based Computer-Supported Collaborative Learning (CSCL) environment. Grounded in sociocultural perspectives, the proposed framework integrates multimodal data sources, including discourse, interaction logs, and linguistic indicators, to assess behavioral, collaborative, and disciplinary engagement through theory-informed human interpretation. Human-annotated engagement ratings serve as ground truth for evaluating whether supervised machine learning models can approximate multimodal interpretive judgments, establishing a pathway toward scalable assessment. Three supervised learning algorithms were compared to examine model performance across engagement dimensions, and multiple regression analysis was conducted to investigate associations between quest-level average engagement and group learning outcomes. Results indicate that average collaborative and disciplinary engagement ratings positively predict group performance, whereas behavioral engagement alone does not significantly relate to learning outcomes. Qualitative analysis of group responses further illustrates these associations. Methodologically and theoretically, the study advances multimodal learning analytics by proposing a generalizable, human-centered workflow that integrates multimodal interpretation with model-assisted scalability, emphasizing the essential role of human interpretation in producing valid and contextually grounded engagement assessments in collaborative learning environments.

Stealth assessment evaluates student competencies using rich interaction data and shows great potential in game-based learning for promoting collaborative problem solving (CPS). However, a key challenge is assessing the impact of algorithmic bias in these models on target populations. To address this challenge, we propose a fairness-centric stealth assessment framework for collaborative game-based learning environments that (1) develops robust models that utilize CPS behaviors for predicting student learning gains, and (2) detects biases within these models. Our findings show that stealth assessment models, when combined with constraint-based sequential pattern mining, effectively predict learning gains. We also identify gender-based disparities in model treatment and demonstrate that common statistical fairness metrics, specifically ABROCA and MADD, capture distinct dimensions of algorithmic bias.

Drawing has been shown to significantly enhance students’ cognitive engagement and learning. However, it has been used infrequently in science classrooms due to practical challenges of assessment. To investigate the potential of multimodal LLMs for automated assessment of students’ science sketches, we created the ScienceSketch sketch-based learning environment for science education. ScienceSketch enables elementary school students ages 9–10 to deepen their science understanding through drawing science models. In ScienceSketch, students draw models representing processes and relationships that describe, explain, and predict natural phenomena, allowing them to communicate scientific concepts. It uses a multimodal large language model (MLLM) to assess student-generated models using a validated rubric. Empirical results from an evaluation of the MMLM-driven sketch assessment framework on elementary students’ science sketches across multiple curricular units demonstrate the models perform best on criteria that correspond to visually explicit or concrete components and worse for more complex criteria that required interpretation of dynamic processes.

Adaptive team training environments support team members as they learn to effectively coordinate their collaborative activities through cross-team communication. Team communication analytics, which informs automated team assessment and coaching in adaptive team training environments, requires accurate and efficient natural language dialogue act recognition (i.e., classifying team members’ utterances to determine their communicative intent). Allowing multiple dialogue act recognition labels for each utterance can support analysis of complex utterances. However, the complexity and variability in team communication pose significant computational challenges for team dialogue act recognition. Large language models (LLMs) show promise for analyzing team communication but require context-specific fine-tuning to capture specialized terminology and each team member’s role-based patterns. In this work, we investigate open-source LLMs for multi-label team dialogue act recognition for team communication analysis. We compare (1) fine-tuning all parameters of a 77-million-parameter Flan-T5-based model and (2) fine-tuning an 8-billion-parameter Llama-3.1-based model using model parameter quantization and low-rank adapters (QLoRA), with bidirectional LSTMs as a baseline. Empirical results show that, when performing dialogue act recognition on spoken dialogue from dialogue-rich medical team training scenarios, LLM-based dialogue act recognition models outperform the non-LLM baseline on all dialogue acts. This work paves the way toward robust team dialogue act recognition for team communication analytics to support adaptive team training environments.

Active learning is widely recognized as an effective approach for improving learning outcomes in introductory programming courses. However, insufficient instructional support often limits students' access to timely, personalized feedback, which is crucial for mastering foundational programming concepts. Although recent advances in AI, particularly large language models, offer scalable opportunities for feedback, concerns about explainability and reliability remain. In this paper, we present an AI-driven classroom assistant that leverages an explainable AI model to analyze student code, map logical errors to instructor-identified misconceptions, and deliver instructor-authored feedback, thereby grounding reliability in instructor-defined pedagogical knowledge. To evaluate the effectiveness of our framework, we conducted an expert evaluation to examine its alignment with instructor-verified feedback and deployed the system in a classroom setting to assess students' perceptions of its usability. Results indicate that the assistant can provide accurate, instructor-verified feedback to students while fostering a positive experience.

As artificial intelligence (AI) becomes increasingly central to healthcare and biomedical research, there is a growing need for learning experiences that help students understand AI concepts while envisioning future career pathways. This paper presents a theory-informed narrative-centered model designed to foster AI literacy and support emerging interest in biomedical careers among early adolescents aged 11-14. Grounded in narrative-centered learning and social cognitive theory, the model articulates how narrative structure, role-based engagement, and consequential decision making can support self-efficacy development, conceptual understanding, and interest in AI-enabled biomedical work. Building on this framework, we describe the design of a narrative-centered educational game in which learners assume the role of a medical intern and investigate patient cases using AI diagnostic tools. We then report findings from a pilot study with 25 students who played the game and participated in focus groups, drawing on gameplay trace data, in-game reflections, and qualitative feedback. Findings suggest high engagement, productive use of AI tools, and evidence of increased awareness of biomedical applications of AI, along with indications of perceived understanding of AI concepts. Together, the theoretical model, game design, and pilot findings illustrate how narrative-centered educational games can serve as research platforms while providing insight into how youth reason about AI in career-connected learning contexts.

Persona prompting is widely used to steer LLM agent behavior, yet the narrative framing of a task can matter more than the assigned persona. We isolate this effect through structural isomorphism, constructing three text-based investigation games that share the same action space, stage progression, and resource constraints while varying only task narrative: disease investigation, IT troubleshooting, and murder mystery. Across 1,890 sessions spanning 3 models and 10 personas, we identify narrative priors: systematic action tendencies activated by a task's story framing, independent of its decision structure. Narrative priors explain 5-31x more behavioral variance than persona, are consistent across model architectures, and in two of three domains are negatively associated with task success. Persona effects that do transfer across narratives arise from behavioral anchors, persona descriptions whose language maps directly onto shared actions. Causal interventions confirm this: removing anchor words from a high-transfer persona reduces cross-narrative consistency by 95

Game-based learning environments (GBLEs) are educational platforms created to promote and scaffold learners’ self-regulated learning (SRL) strategies while learning complex STEM topics. However, learners can still abuse the intended purposes of GBLEs and engage in ‘gaming the system’, deliberate behaviors employed to forgo the learning process while still completing the task at hand. ‘Gaming the system’ is often investigated within intelligent tutoring systems (ITSs), yet these behaviors have been the subject of limited investigation in narrative-based GBLEs. Thus, this study identified two ‘gaming the system’ behaviors from undergraduate students’ (N = 93) gameplay during the narrative-based GBLE, Crystal Island, and examined how their prior knowledge of microbiology and two agency conditions influenced their frequencies of gaming behaviors and their learning outcomes. Results indicated that students with high prior knowledge engaged in significantly more ‘trial-and-error’ gaming behaviors than those with lower prior knowledge. Moreover, we found that students with partial agency engaged in significantly more “trial-and-error’ gaming behaviors than those will full agency; whereas those will full agency engaged in more ‘guessing’ gaming behaviors than those with full agency. These results indicate the importance for identifying contextually driven gaming behaviors, as well as future GBLE scaffolds to adopt real-time, adaptive scaffolding methods based on learner’s active gameplay and SRL (in)efficiency.