With advances in artificial intelligence (AI), educational researchers have integrated AI into mathematics education to offer scalable instructional practices and personalized learning. One such innovation is teachable AI agents, designed as learners to facilitate learning by teaching. Previous research has evidenced the benefits of learning by teaching, and its effectiveness depends on the quality of tutor-tutee interaction. However, few studies have explored how features of teachable agents, particularly personality traits, influence student interactions and the agents’ effectiveness. Given the documented importance of personality traits in student learning, this empirical study examines the relationship between teachable AI agents’ personality traits and students’ math learning experiences in a naturalistic setting. Results indicated that students provided more cognitive support when interacting with teachable agents characterized by neuroticism, openness, and conscientiousness, while more affect management and non-responsive behaviors were observed with agents displaying extraversion. These interaction patterns impacted the effectiveness of the teachable agents, providing implications for the integration of AI systems into education.

The importance and challenge of Algebra learning is widely recognized, with students across the U.S. facing difficulties due to the subject’s complexity. While extensive research has focused on enhancing Algebra learning in K-12 education, the reusability, scalability, and effectiveness of the strategies employed (e.g., manual interventions and digital tutoring platforms) remain limited. Conversational AI (ConvAI), enabled by the advancement of large language models (LLMs), emerges as a potential tool for automatic, personalized, and effective student support. However, ethical concerns surrounding diversity, safety, sentiment, and stereotype associated with ConvAI are prominent, and empirical studies examining its application in education are scarce. The purpose of this study is to develop a ConvAI system that mitigates the potential ethical concerns and empirically evaluate the effect of such a system for math learning. Specifically, we first examined computational strategies to mitigate the ethical concerns of ConvAI in educational setting with educational big data (npretraining = 2,097,139) and found that researchers could effectively enhance ConvAI responsibility through the investigated algorithmic strategies. Then, a ConvAI system was constructed using these strategies, guided by learning sciences principles. Lastly, we examined students’ eye-tracking patterns, acceptance, and learning processes when using this ConvAI system to learn Algebra through a random experiment (nparticipant = 151). Participants using the developed ConvAI demonstrated generally increased visual attention levels as compared to the control group. Moreover, participants expressed a positive acceptance towards the ConvAI technology. Finally, participants’ interaction patterns with the ConvAI technology influenced their Algebra learning. These results provide insights for both educational researchers and practitioners to integrate ConvAI in learning environments.

Supporting learners in achieving high-level socially shared regulation of learning (SSRL) in the online collaborative learning (OCL) context presents challenges that the utilization of artificial intelligence (AI) technologies may help solve. However, the effective uses of AI to support multifaceted areas (cognition, metacognition, and motivation) and phases (forethought, performance, and reflection) of SSRL remain elusive. Furthermore, research on developing an educational AI and what pedagogical attributes and elements are required for AI to support students' SSRL effectively is limited. This study, therefore, aims to investigate students' perceptions of AI applications in enhancing SSRL and to explore the essential pedagogical elements necessary for AI to support SSRL during the OCL. To achieve these aims, the study conducted Focus Group Interviews facilitated by 9 scenarios of AI application storyboards and paper prototypes with 30 undergraduate and graduate students. The study findings show that students perceive various types of AI to support cognitive, metacognitive, and motivational areas across different SSRL phases. The study also found that students viewed AI as an active learning agent, serving in roles previously inhabited solely by human educators and students. Furthermore, the study reveals seven key pedagogical elements across TPACK components such as pedagogical, content, technological, pedagogical content, technological pedagogical, technological content, and technological pedagogical content knowledge deemed crucial by students for AI to support SSRL in OCL effectively. These findings offer implications for using and designing educationally relevant AI to support SSRL in OCL environments.

While mathematical creative writing can engage students in expressing mathematical ideas in an imaginative way, young learners struggle in this process. Generative AI (GenAI) offers possibilities for supporting them, such as providing story generation. As the design of GenAI-powered learning technologies requires careful consideration of the technology reception in classrooms, this study explores students' and teachers' perceptions of creative mathematical writing with the developed GenAI-powered technology. We adopted a qualitative thematic analysis of the interviews with 10 elementary school students and six teachers, triangulated with open-ended survey responses and classroom observation of 79 students. We identified six themes and 19 subthemes regarding benefits and challenges; perceived benefits include fostering creativity and understanding in math, AI literacy, and affective domain, while students- and AI-related challenges were identified as well. This study contributes by investigating the lived experience of GenAI-supported learning and the design considerations for GenAI-powered learning technologies and instructions.

As conversational AI apps such as Siri and Alexa become ubiquitous among children, the CS education community has begun leveraging this popularity as a potential opportunity to attract young learners to AI, CS, and STEM learning. However, teaching conversational AI to K-12 learners remains challenging and unexplored due in part to the abstract and complex nature of some conversational AI concepts, such as intents and training phrases. One promising approach to teaching complex topics in engaging ways is through unplugged activities, which have been shown to be highly effective in fostering CS conceptual understanding without using computers. Research efforts are underway toward developing unplugged activities for teaching AI, but few thus far have focused on conversational AI. This experience report describes the design and iterative refinement of a series of novel unplugged activities for a conversational AI summer camp for middle school learners. We discuss learner responses and lessons learned through our implementation of these unplugged activities. Our hope is that these insights support CS education researchers in making conversational AI learning more engaging and accessible to all learners.

During collaborative learning, confusion and conflict emerge naturally. However, persistent confusion or conflict have the potential to generate frustration and significantly impede learners' performance. Early automatic detection of confusion and conflict would allow us to support early interventions which can in turn improve students' experience with and outcomes from collaborative learning. Despite the extensive studies modeling confusion during solo learning, there is a need for further work in collaborative learning. This paper presents a multimodal machine-learning framework that automatically detects confusion and conflict during collaborative learning. We used data from 38 elementary school learners who collaborated on a series of programming tasks in classrooms. We trained deep multimodal learning models to detect confusion and conflict using features that were automatically extracted from learners' collaborative dialogues, including (1) language-derived features including TF-IDF, lexical semantics, and sentiment, (2) audio-derived features including acoustic-prosodic features, and (3) video-derived features including eye gaze, head pose, and facial expressions. Our results show that multimodal models that combine semantics, pitch, and facial expressions detected confusion and conflict with the highest accuracy, outperforming all unimodal models. We also found that prosodic cues are more predictive of conflict, and facial cues are more predictive of confusion. This study contributes to the automated modeling of collaborative learning processes and the development of real-time adaptive support to enhance learners' collaborative learning experience in classroom contexts.

As artificial intelligence (AI) becomes more prominent in children's lives, an increasing number of researchers and practitioners underscored the importance of integrating AI as learning content in K-12. Despite the recent efforts in developing AI curricula and guiding frameworks in AI education, the educational opportunities often do not provide equally engaging and inclusive learning experiences for all learners. To promote equality and equity in society and increase competitiveness in the AI workforce, it is essential to broaden participation in AI education. However, the framework that guides teachers and learning designers into inclusive learning design tailored for AI education is lacking. Universal Design for Learning (UDL) provides guidelines for making learning more inclusive across disciplines. Based on the principles of UDL, this paper proposes a framework to guide the design of inclusive AI learning. We conducted a systematic literature review to identify AI learning design-related articles and synthesized them into our proposed framework. Our new framework includes the core component of AI learning content (i.e., five big ideas), anchored by the three UDL principles (the “why,” “what,” and “how” of learning), and six praxes with pedagogical examples of AI education. Alongside this, we present an illustrative example of the application of our proposed framework in the context of a middle school AI summer camp. We hope this paper will guide researchers and practitioners in designing more inclusive AI learning experiences.

As more clinical workflows continue to be augmented by artificial intelligence (AI), AI literacy among physicians will become a critical requirement for ensuring safe and ethical AI-enabled patient care. Despite the evolving importance of AI in healthcare, the extent to which it has been adopted into traditional and often-overloaded medical curricula is currently unknown. In a scoping review of 1,699 articles published between January 2016 and June 2024, we identified 18 studies which propose guiding frameworks, and 11 studies documenting real-world instruction, centered around the integration of AI into medical education. We found that comprehensive guidelines will require greater clinical relevance and personalization to suit medical student interests and career trajectories. Current efforts highlight discrepancies in the teaching guidelines, emphasizing AI evaluation and ethics over technical topics such as data science and coding. Additionally, we identified several challenges associated with integrating AI training into the medical education program, including a lack of guidelines to define medical students AI literacy, a perceived lack of proven clinical value, and a scarcity of qualified instructors. With this knowledge, we propose an AI literacy framework to define competencies for medical students. To prioritize relevant and personalized AI education, we categorize literacy into four dimensions: Foundational, Practical, Experimental, and Ethical, with tailored learning objectives to the pre-clinical, clinical, and clinical research stages of medical education. This review provides a road map for developing practical and relevant education strategies for building an AI-competent healthcare workforce.

Previous literature has associated math literacy with linguistic factors such as verbal ability and phonological skills. However, few studies have investigated linguistic synchrony, shown in mathematical discussions. This study modelled math literacy and examined the relationship of math literacy with linguistic synchrony between students and facilitators. We retrieved data from 20,776 online mathematical discussion threads at a secondary school level. First, we assessed students' math literacy based on their discussions and classified them into high- and low-math literacy groups. Then, we conducted Cross-Recurrence Quantification Analysis (CRQA) to calculate linguistic synchrony within each thread. The result implies that students with high math literacy are more likely to share common words (eg, mathematical terms) with facilitators. At the same time, they would paraphrase the facilitators' words rather than blindly mimic them as the exact sentences or phrases. On the other hand, students with low math literacy tend to use overlapping words with facilitators less frequently and are more likely to repeat the exact same phrases from the facilitators. The findings provide an empirical data analysis and insights into mathematical discussions and linguistic synchrony. In addition, this paper implies the directions to improve online mathematical discussions and foster math literacy.Practitioner notes What is already known about this topic Mathematical discussions are known to be an effective way to promote math literacy. Math literacy and linguistic skills have a strong link. Linguistic synchrony is related to better collaboration and common knowledge building. What this paper adds Reveals the relationship between math literacy and linguistic synchrony and deepens the understanding of digital communication in online learning environments. Provides empirical analysis of natural language data in group discussions using CRQA. Conceptualizes linguistic synchrony with three sub-concepts: linguistic concurrence, predictability, and complexity. Implications for practice and/or policy Educators and practitioners could utilize the automatic formative assessment of math literacy based on the student's language use in mathematical discussions. Educational technology researchers and designers could include CRQA indices and recurrence plots in the dashboard design to provide information to support teachers and learners. Teachers would be able to provide real-time interventions to promote effective mathematical communication and foster math literacy throughout mathematical discussions.

This work-in-progress research paper aims to explore students' dropout behavior during video engagement in online learning platforms. As online learning becomes increasingly popular, analyzing how students engage with video content provides important insights into their learning behaviors. This study explores multiple factors influencing K-12 students' in-video dropout rates in online math education. We examined 34,666,481 log entries from Math Nation, covering 1313 videos and 14,251 students. Using survival analysis, we evaluated how 27 variables, including demographic details, video interaction behaviors, and video characteristics(e.g. length, category), affect in-video dropout. Our findings reveal that video length significantly predicts dropout, with each additional minute increasing the dropout rate by 1.26%. Videos with higher dropout rates often feature more frequent pauses, jumps, and rewatches. The study also highlights that the quality of video content, the creators of the videos, and how students interact with the videos are crucial factors affecting dropout rates. Further research is needed to determine the specific causes of video dropout.

While virtual learning environments (VLEs) are widely used in K-12 education for classroom instruction and self-study, young students' success in VLEs highly depends on their self-regulated learning (SRL) skills. Therefore, it is important to provide personalized support for SRL. One important precursor of designing personalized SRL support is to understand students' SRL behavioral patterns. Extensive studies have clustered SRL behaviors and prescribed personalized support for each cluster. However, limited attention has been paid to the algorithm bias and fairness of clustering results. In this study, we "fairly" clustered the behavioral patterns of SRL using fair-capacitated clustering (FCC), an algorithm that incorporates constraints to ensure fairness in the assignment of data points. We used data from 14,251 secondary school learners in a virtual math learning environment. The results of FCC showed that it could capture six clusters of SRL behaviors in a fair way; three clusters belonging to high-performing (i.e., H-1. Help-provider, H-2) Active SRL learner, H-3) Active onlooker), and three clusters in low-performing groups (i.e., L-1) Quiz-taker, L-2) Dormant learner, and L-3) Inactive onlooker). The findings provide a better understanding of SRL patterns in online learning and can potentially guide the design of personalized support for SRL.

Ongoing advancements in generative AI (GenAI) have boosted the potential of applying long-standing "learning-by-teaching" practices in the form of a teachable agent (TA). Despite the recognized roles and opportunities of TAs, less is known about how GenAI could create synergy or introduce challenges in TAs and how students perceived the application of GenAI in TAs. This study explored middle school students' perceived roles, benefits, and challenges of GenAI-powered TAs in an authentic mathematics classroom. Through classroom observation, focus-group interviews, and open-ended surveys of 108 sixth-grade students, we found that students expected the GenAI-powered TA to serve as a learning companion, facilitator, and collaborative problem-solver. Students also expressed the benefits and challenges of GenAI-powered TAs. This study provides implications for the design of educational AI and AI-assisted instruction.

BackgroundStealth assessment is a learning analytics method, which leverages the collection and analysis of learners' interaction data to make real-time inferences about their learning. Employed in digital learning environments, stealth assessment helps researchers, educators, and teachers evaluate learners' competencies and customize the learning experience to their specific needs. This adaptability is closely intertwined with theories related to learning, engagement, and motivation. The foundation of stealth assessment rests on evidence-cantered design (ECD), consisting of four core models: the Competency Model (CM), Evidence Model, Task Model, and Assembly Model.ObjectiveThe first step in designing a stealth assessment entails producing operational definitions of the constructs to be assessed. The CM establishes a framework of latent variables representing the target constructs, as well as their interrelations. When developing the CM, assessment designers must produce clear descriptions of the claims associated with the latent variables and their states, as well as sketch out how the competencies can be measured using assessment tasks. As the designers elaborate on the assessment model, the CM definitions need to be revisited to make sure they work with the scope and constraints of the assessment. Although this is the first step, problems at this stage may result in an assessment that does not meet the intended purpose. The objective of this paper is to elucidate the necessary steps for CM development and to highlight potential challenges in the process, along with strategies for addressing them, particularly for designers without much formal assessment experience.MethodThis paper is a methodological exposition, showcasing five examples of CM development. Specifically, we conducted a qualitative retrospective analysis of the CM development procedure, wherein participants unfamiliar with ECD applied the framework and showcased their work. In a stealth assessment course, four groups of students (novice stealth assessment designers) engaged in developing stealth assessments for challenging-to-measure constructs across four distinct projects. During their CM development process, we observed various activities to pinpoint areas of difficulty.ResultsThis paper presents five illustrative examples, including one for assessing physics understanding and four for the development of CMs for four complex competencies: (1) systems thinking, (2) online information credibility evaluation, (3) computational thinking, and (4) collaborative creativity. Each example represents a case in CM development, offering valuable insights.ConclusionThe paper concludes by discussing several guidelines derived from the examples discussed. Emphasizing the importance of dedicating ample time to fine-tune CMs can significantly enhance the accuracy of assessments related to learners' knowledge and skills. It underscores the significance of qualitative phases in crafting comprehensive stealth assessments, such as CMs, alongside the quantitative statistical modeling and technical aspects of these assessments. What is currently known about this topic? Stealth assessment represents an unobtrusive, automated formative assessment method. This method uses learning analytics within digital learning environments (e.g., games). The main purpose is to assess and foster the competencies of diverse learners.What does this paper add? This paper serves as a conceptual and methodological guide. This paper focuses on the critical process of competency model development. Competency model development is a crucial step in the creation of stealth assessments.Implications for practice/or policy Learning scientists and assessment designers can leverage this paper as a resource. Assessment designers can benefit from this paper and see various examples of the process.

Although researchers recognize the importance of discussing support for math learning within online learning communities, there is a lack of relevant network classifying methods and analyses at the group level to understand the behavioral differences between groups with varying levels of activity, including their mathematical literacies. In this research, we investigated different groups within a large asynchronous online discussion community for middle school students, focusing on their interaction patterns and the quality of their mathematical engagement. First, we employed an extended Surprise detection algorithm that evaluates interaction quality to classify users into core, periphery, and extra-periphery groups. Following this classification, we performed social network analysis to understand the interaction patterns among these groups. For discourse analysis, we used topic modeling methods to analyze the socio-semantic network structure of the discussions. To assess differences in math literacy and discussion success rates among the groups, we applied the Mann-Whitney U test. Findings indicate that each group is more responsive to its members, with the core group demonstrating a balanced response pattern. X-periphery students primarily engage in casual chats and open queries, indicating a more focused participation aimed at immediate learning needs. Notably, the X-periphery group exhibits the highest math literacy and discussion success rates, suggesting that lower activity levels do not hinder communication efficiency. These findings highlight the importance of considering group dynamics and roles in designing online math learning activities to foster effective communication and support, offering practical insights for sustaining online learning communities through tailored discussion activities.

Summer camps have become popular for introducing K-12 learners to computer science (CS) and artificial intelligence (AI) in informal learning environments.Facilitators play crucial roles in guiding and engaging learners in these contexts, but there is limited research on their roles in informal AI learning.This paper examines facilitators' dialogues with campers in a middle school AI summer camp, identifying eight major facilitator roles.The roles differed depending on group dynamics and project phase.The paper provides empirical grounding to define facilitators' roles in AI learning and guide the design of professional development for camp facilitators.

Mathematical discussions have become a popular educational strategy to promote math literacy. While some studies have associated math literacy with linguistic factors such as verbal ability and phonological skills, no studies have examined the relationship between linguistic synchrony and math literacy. In this study, we modeled linguistic synchrony and students’ math literacy from 20,776 online mathematical discussion threads between students and facilitators. We conducted Cross-Recurrence Quantification Analysis (CRQA) to calculate linguistic synchrony within each thread. The statistical testing result comparing CRQA indices between high and low math literacy groups shows that students with high math literacy have a significantly higher Recurrence Rate (RR), Number of Recurrence Lines (NRLINE), and the average Length of lines (L), but lower Determinism (DET) and normalized Entropy (rENTR). This result implies that students with high math literacy are more likely to share common words with facilitators, but they would paraphrase them. On the other hand, students with low math literacy tend to repeat the exact same phrases from the facilitators. The findings provide a better understanding of mathematical discussions and can potentially guide teachers in promoting effective mathematical discussions.

Conversational AIs such as Alexa and ChatGPT are increasingly ubiquitous in young people’s lives, but these young users are often not afforded the opportunity to learn about the inner workings of these technologies. One of the most powerful ways to foster this learning is to empower youth to create AI that is personally and socially meaningful to them. We have built a novel development environment, AMBY–“AI Made By You”–for youth to create conversational agents. AMBY was iteratively designed with and for youth aged 12–13 through contextual inquiry and usability studies. AMBY is designed to foster AI learning with features that enable users to generate training datasets and visualize conversational flow. We report on results from a two-week summer camp deployment, and contribute design implications for conversational AI authoring tools that empower AI learning for youth.