Context-sensitive professional development for emerging educational technologies is often lacking, leaving many teachers without adequate support for classroom integration. Although recent advances in generative AI (GenAI) have prompted growing interest in AI-based support tools for teacher learning, more research is needed on how such tools are interpreted and integrated into classroom practice. We examined how secondary mathematics teachers engaged with an AI-powered support bot (a custom GPT), embedded in an asynchronous training module on interactive simulation tools. Using qualitative analyses of open-ended responses within the module, we explored how teachers positioned the support bot within their instructional planning and sense-making processes while responding to hypothetical classroom scenarios. Teachers’ engagement with the AI tool was shaped by their familiarity of teaching with simulation tools, instructional needs, and professional orientations toward GenAI, highlighting the importance of teacher agency in understanding and designing for technology integration in authentic learning environments. We discuss implications for future research, as well as the design of AI-supported teacher training environments.

Lexical alignment occurs when conversational partners converge on similar linguistic patterns. In collaborative learning settings, lexical alignment could indicate rapport, which can further predict learning and the collaborators' evolving shared understanding. Traditional approaches to alignment computation often focus more on the summary statistics computed at the end of the conversation, which usually do not capture the conversational dynamics effectively. This work investigates how alignment evolves in a conversation by modeling lexical alignment trajectories between human dyads while interacting with a teachable robot. We find that, along with the summary statistics, the alignment curve parameters and the time taken to reach key alignment moments significantly predict rapport. We further see significant relationships between the early turns in the conversation and the overall alignment trajectories, indicating the importance of modeling conversational dynamics to plan real-time interventions in the robot, with the goal of altering the alignment trajectories and, consequently, learning outcomes.

Science educators incorporate collaborative engagement in model-based argumentation to meet curricular goals and build students' capacity for scientific epistemic and social practices. During collaboration, groups encounter various challenges (e.g., lack of task understanding) and engage in social regulation to overcome them. However, little is known about whether and how various contextual factors (e.g., teacher presence, classroom climate, or academic discipline) can influence the nature of students' collaboration and social regulation. The purpose of our qualitative case study was to examine how one such contextual factor, teacher presence, related to high school students' discourse interactions and social regulation of learning during a collaborative model-based scientific argumentation task. In one classroom, the teacher was continuously present during groups' discussions. In the other classroom, the teacher was intermittently present. We found that groups with a continuously present teacher had high on-task engagement that was teacher-led, with the students relying on the teacher for regulation. Groups with intermittent teacher presence had more off-task interactions but also engaged in more dialogic argumentation discourse with each other, initiating and enacting more modes of social regulation of learning than the other groups. These findings suggest that teachers must thoughtfully manage their presence and absence to instruct, model, scaffold, and fade their support for both scientific argumentation and the social regulation skills necessary to productively enact that argumentation, intentionally varying emphasis on one or the other. These findings highlight the importance of future research on teacher presence and other contextual factors that can affect how students collaborate and learn.

While speech-enabled teachable agents have some advantages over typing-based ones, they are vulnerable to errors stemming from misrecognition by automatic speech recognition (ASR). These errors may propagate, resulting in unexpected changes in the flow of conversation. We analyzed how such changes are linked with learning gains and learners' rapport with the agents. Our results show they are not related to learning gains or rapport, regardless of the types of responses the agents should have returned given the correct input from learners without ASR errors. We also discuss the implications for optimal error-recovery policies for teachable agents that can be drawn from these findings.

Gaming the system, a behavior in which learners exploit a system's properties to make progress while avoiding learning, has frequently been shown to be associated with lower learning. However, when we applied a previously validated gaming detector across conditions in experiments with an algebra tutor, the detected gaming was not associated with reduced learning, challenging its validity in our study context. Our exploratory data analysis suggested that varying contextual factors across and within conditions contributed to this lack of association. We present a new approach, latent variable-based gaming detection (LV-GD), that controls for contextual factors and more robustly estimates student-level latent gaming tendencies. In LV-GD, a student is estimated as having a high gaming tendency if the student is detected to game more than the expected level of the population given the context. LV-GD applies a statistical model on top of an existing action-level gaming detector developed based on a typical human labeling process, without additional labeling effort. Across three datasets, we find that LV-GD consistently outperformed the original detector in validity measured by association between gaming and learning as well as reliability. LV-GD also afforded high practical utility: it more accurately revealed intervention effects on gaming, revealed a correlation between gaming and perceived competence in math and helped understand productive detected gaming behaviors. Our approach is not only useful for others wanting a cost-effective way to adapt a gaming detector to their context but is also generally applicable in creating robust behavioral measures.

Speakers build rapport in the process of aligning conversational behaviors with each other. Rapport engendered with a teachable agent while instructing domain material has been shown to promote learning. Past work on lexical alignment in the field of education suffers from limitations in both the measures used to quantify alignment and the types of interactions in which alignment with agents has been studied. In this paper, we apply alignment measures based on a data-driven notion of shared expressions (possibly composed of multiple words) and compare alignment in one-on-one human-robot (H-R) interactions with the H-R portions of collaborative human-human-robot (H-H-R) interactions. We find that students in the H-R setting align with a teachable robot more than in the H-H-R setting and that the relationship between lexical alignment and rapport is more complex than what is predicted by previous theoretical and empirical work.

Despite recent increases in research on emotions and regulation in collaborative learning, measuring both constructs remains challenging and often lacks structure. Researchers need a systematic method to measure both the formation of emotions and subsequent regulation in collaborative learning environments. Drawing from the Formation and Regulation of Emotions in Collaborative Learning (FRECL) model, I introduce a new observational coding procedure that provides comprehensive guidelines for coding these phenomena. The FRECL coding procedure has been implemented successfully in other studies and is described here in detail. Specifically, I detail the ideal situations for using the procedure, discuss background information and present a codebook and empirical examples for each stage of the FRECL model, and provide additional considerations that allow researchers flexibility based on their own experiences and preferences. This procedure extends past research by providing an accessible observational protocol that is both systematic and comprehensive. The FRECL coding procedure can benefit future research by providing more organized consistency to the measurement of collaborative emotions and regulation.

Gaming the system, a behavior in which learners exploit a system’s properties to make progress while avoiding learning, has frequently been shown to be associated with lower learning. However, when we applied a previously validated gaming detector across conditions in experiments with an algebra tutor, the detected gaming was not associated with learning, challenging its construct validity. Our iterative exploratory data analysis suggested that some contextual factors that varied across and within conditions might contribute to this lack of association. We present a latent variable model, item response theory-based gaming detection (IRT-GD), that accounts for contextual factors and estimates latent gaming tendencies as the degree of deviation from normative behaviors across contexts. Item response theory models, widely used in knowledge assessment, account for item difficulty in estimating latent student abilities: students are estimated as having higher ability when they can get harder items correct than when they only get easier items correct. Similarly, IRT-GD accounts for contextual factors in estimating latent gaming tendencies: students are estimated as having a higher gaming tendency when they game in less commonly gamed contexts than when they only game in more commonly gamed contexts. IRT-GD outperformed the original detector on three datasets in terms of the association with learning. IRT-GD also more accurately revealed intervention effects on gaming and revealed a correlation between gaming and perceived competence in math. Our approach is not only useful for others wanting to apply a gaming assessment in their context but is also generally applicable in creating robust behavioral measures.

Working collaboratively in groups can positively impact performance and student engagement. Intelligent social agents can provide a source of personalized support for students, and their benefits likely extend to collaborative settings, but it is difficult to determine how these agents should interact with students. Reinforcement learning (RL) offers an opportunity for adapting the interactions between the social agent and the students to better support collaboration and learning. However, using RL in education with social agents typically involves training using real students. In this work, we train an RL agent in a high-quality simulated environment to learn how to improve students’ collaboration. Data was collected during a pilot study with dyads of students who worked together to tutor an intelligent teachable robot. We explore the process of building an environment from the data, training a policy, and the impact of the policy on different students, compared to various baselines.

Given the rapid pace of technological change, access to unlimited information, and diverse forms of complex text, the importance and demand for enhanced literacy skills is greater than ever. Accordingly, researchers have begun developing integrated, multifaceted interventions that dynamically support enhanced literacy competence. The purpose of this year-long quasi-experimental study was to compare the effects of a multifaceted, rigorous discussion intensive literacy intervention called Quality Talk (QT) to a comparison intervention. Fourth- and fifth-grade students (QT treatment, n = 133; comparison, n = 155) from two public schools participated in a district-wide literacy program with half also participating in QT within their language arts class. Spanning baseline and two subsequent time points, findings revealed that, on average, students evidenced statistically significant increases on one form of basic-level comprehension performance over time with no statistically significant difference between QT and comparison classrooms. Given that treatment and comparison classrooms engaged in a district-wide literacy initiative with supplemental daily literacy instruction, these results are not altogether unexpected. However, despite the enhanced literacy instruction across all classes, from Time 2 to Time 3, growth in QT students' high-level comprehension, as measured via written argumentation essay performance, was statistically significantly greater than their comparison peers' growth. This study informs the future of education research and practice regarding the feasibility and utility of relevant and rigorous multifaceted literacy interventions focused upon small-group discussion.

Teaching others has been shown to be an activity in which students can learn new information in both human-human (peer-tutoring) and human-computer interactions (teachable robots). One factor that may help foster learning and engagement when teaching others is the development of positive rapport and perceptions between the tutor, tutee, and robot. However, it is not clear what factors might affect the development of rapport. We explore whether having two students work together with a teachable robot might facilitate positive perceptions of the robot, rapport-building, and positive learning outcomes. In an exploratory pilot study, students were assigned to either work together in dyads (n = 28) or individually (n = 12) to help a teachable robot (Emma) solve math problems. Preliminary results showed that those who worked in a dyad had generally more positive perceptions of the robot than those who worked individually. These benefits were not observed for rapport where there were few differences between dyads and individuals, or learning where there was no difference on the posttest. We discuss the implications of these results for future research to explore the potential benefits of collaborative teaching of a robot learner.

Teachers, schools, districts, states, and technology developers endeavor to personalize learning experiences for students, but definitions of personalized learning (PL) vary and designs often span multiple components. Variability in definition and implementation complicate the study of PL and the ways that designs can leverage student characteristics to reliably achieve targeted learning outcomes. We document the diversity of definitions of PL that guide implementation in educational settings and review relevant educational theories that could inform design and implementation. We then report on a systematic review of empirical studies of personalized learning using PRISMA guidelines. We identified 376 unique studies that investigated one or more PL design features and appraised this corpus to determine (1) who studies personalized learning; (2) with whom, and in what contexts; and (3) with focus on what learner characteristics, instructional design approaches, and learning outcomes. Results suggest that PL research is led by researchers in education, computer science, engineering, and other disciplines, and that the focus of their PL designs differs by the learner characteristics and targeted outcomes they prioritize. We further observed that research tends to proceed without a priori theoretical conceptualization, but also that designs often implicitly align to assumptions posed by extant theories of learning. We propose that a theoretically guided approach to the design and study of PL can organize efforts to evaluate the practice, and forming an explicit theory of change can improve the likelihood that efforts to personalize learning achieve their aims. We propose a theory-guided method for the design of PL and recommend research methods that can parse the effects obtained by individual design features within the "many-to-many-to-many" designs that characterize PL in practice.

Computer tutor data indicate that more learning opportunities yield greater achievement, but also confirm there are gaps in the number and quality of opportunities marginalized students receive that technology alone does not address. Personalized learning with mentors can close this gap in opportunities but is expensive to implement. We introduce a free, web-based application, Personalized Learning2 (PL2), designed to improve mentoring efficiency by connecting mentors to intervention and instructional resources. Preliminary findings indicated that PL2 's categorization of students based on math learning software data enabled mentors to focus their efforts, and that mentors found PL2 resources to positively expand how they taught and mentored.

Analytics of student learning data are increasingly important for continuous redesign and improvement of tutoring systems and courses. There is still a lack of general guidance on converting analytics into better system design, and on combining multiple methods to maximally improve a tutor. We present a multi-method approach to data-driven redesign of tutoring systems and its empirical evaluation. Our approach systematically combines existing and new learning analytics and instructional design methods. In particular, our methods involve identifying difficult skills and creating focused tasks for learning these difficult skills effectively following content redesign strategies derived from analytics. In our past work, we applied this approach to redesigning an algebraic modeling unit and found initial evidence of its effectiveness. In the current work, we extended this approach and applied it to redesigning two other tutor units in addition to a second iteration of redesigning the previously redesigned unit. We conducted a one-month classroom experiment with 129 high school students. Compared to the original tutor, the redesigned tutor led to significantly higher learning outcomes, with time mainly allocated to focused tasks rather than original full tasks. Moreover, it reduced over- and under-practice, yielded a more effective practice experience, and selected skills progressing from easier to harder to a greater degree. Our work provides empirical evidence of the effectiveness and generality of a multi-method approach to data-driven instructional redesign.

Objective. To explore pharmacists' and pharmacy students' perceptions regarding the significance of changing the features of test item scenario (eg, switching from a health care to a non-health care context) on their situational judgment test (SJT) responses.Methods. Fifteen Doctor of Pharmacy students and 15 pharmacists completed a 12-item SJT intended to measure empathy. The test included six scenarios in a health care context and six scenarios in a non-health care context; participants had to rank potential response options in order of appropriateness and no two items could be of equal rank. Qualitative data were collected individually from participants using think-aloud and cognitive interview techniques. During the cognitive interview, participants were asked how they selected their final responses for each item and whether they would have changed their answer if features of the scenario were switched (eg, changed to a non-health care context if the original item was in a health care context). Interviews were transcribed and a thematic analysis was conducted to identify the features of the scenario for each item that were perceived to impact response selections.Results. Participants stated that they would have changed their responses on average 51.3% of the time (range 20%-100%) if the features of the scenario for an item were changed. Qualitative analysis identified four pertinent scenario features that may influence response selections, which included information about the examinee, the actors in the scenario, the relationship between examinee and actors, and details about the situation. There was no discernible pattern linking scenario features to the component of empathy being measured or participant type.Conclusion. Results from this study suggest that the features of the scenario described in an SJT item could influence response selections. These features should be considered in the SJT design process and require further research to determine the extent of their impact on SJT performance.

INTRODUCTION:The ability to identify criteria (ATIC) refers to an examinee's capacity to distinguish the construct being evaluated. Previous research indicates ATIC can be predictive of performance on some assessments. This exploratory study investigated the relationship between a participant's ability to identify criteria and their performance on an empathy situational judgment test (SJT), an assessment format used to measure social and behavioral attributes.METHODS:A 12-item empathy SJT was completed by 15 students and 15 pharmacists. During a cognitive interview, participants were asked what they believed each exam question measured. Responses were coded to determine whether participants stated "empathy" (indication of ATIC). The point-biserial correlation coefficient was calculated to explore the relationship of ATIC (correctly or not correctly identifying the item measured empathy) and performance on the SJT (total score).RESULTS:Participants identified empathy 33.3% of the time, and it was the construct most often identified. Pharmacists (27.5%) identified empathy less often than students (39.2%). When empathy was identified as the construct, it was most often reported for items in a non-healthcare setting (56.3%) rather than a healthcare setting (43.7%) and for questions targeting affective empathy (71.3%) rather than cognitive empathy (28.7%). There were no statistically significant relationships with correctly identifying the construct and performance on individual items and the overall test.CONCLUSIONS:There is inconclusive evidence that ATIC relates to performance on an empathy SJT. Additional research is needed to evaluate the role of ATIC and assessment performance to corroborate study results.

A key strategy to being successful in math is help-seeking. Much research has focused on how students seek help from teachers, but students also benefit from peer assistance. We analyzed conversations from eight focus groups related to students’ paths to success in math. We found that students frequently brought up peer interactions as critical to their success. Three emergent themes were: support versus comparison, balance of effort within study groups, and friends versus peers. Our findings qualitatively extend current research by highlighting key ideas that impact peer interactions, such as increased reluctance to seek help in larger classes. Our findings have implications for classroom and group structures, including encouraging peer help-seeking and help-giving, as well as addressing belonging and sense of community. Introduction and background As students learn, they may face challenges that require assistance to overcome. However, in the face of adversity, students do not always seek help, which can inhibit their academic achievement. In school environments, students are shaped by their interactions with peers. With more detailed accounts of how peer interactions can aid or harm students’ paths to math success, educators can orient their classrooms to set students up for math achievement, potentially through peer interactions and collaboration. While the current literature focuses on quantitative patterns, this study seeks to learn more about how students describe their lived experiences. Seeking help when navigating obstacles to learning is a key self-regulated learning skill (Karabenik, 2011). Although students tend to focus on teachers for learning support, using peers as a resource has additional benefits. Access to multiple explanations (i.e., beyond that of the teacher or text) of mathematical concepts can help students experience new perspectives (Tripathi, 2008). Moreover, helping others and seeing peers model success can increase self-efficacy and ownership in learning (Walker et al., 2010). Help-seeking from peers can also increase socialization, an important developmental skill for younger students (Newman, 2000). Despite the benefits, some students still avoid seeking help for many reasons. For some, the likelihood of seeking help is negatively related to the perception of subsequent psychological risks (Peeters et al., 2020). These can include embarrassment in admitting confusion or errors (Karabenik, 2011), fear of frustrating the teacher or slowing down the class (Peeters et al., 2020), indebtedness to the help-givers (Karabenik, 2011), and negative social comparison to other more-capable peers (Newman & Schwager, 1993). Additionally, if students lack confidence in their peers’ (or even the teacher’s) ability to understand their confusion and provide adequate help, they will likely not seek help (Newman, 2000; Peeters et al., 2020). Students’ help-seeking behaviors and peer interactions are also related to their academic achievement goals (Roussel et al., 2011; Shim & Finch, 2014). Compared to students focused on demonstrating competence in relation to the task itself or their own previous competence (mastery goals), students focused on demonstrating competence in relation to others (performance goals) are less likely to engage in productive help-seeking behaviors or see peers as instructional and emotional supports (Shim & Finch, 2014). Naturally, when students focus on their own learning and improvement, rather than in relation to others, they are less likely to fear social comparison (Roussel et al., 2011) or engage in competitive, maladaptive behaviors’ (Newman & Schwager, 1993). Most studies on peers as a resource are quantitative, focusing on identifying variables that impact helpseeking behavior and learning (e.g., Roussel et al., 2011). More research is needed to understand how students think about interactions during peer support and how these can help them overcome challenges while learning math. Therefore, in this qualitative study, our research question is: How do students describe interactions with peers on their paths to math success?

We report on a design-based research study that was conducted over two years. We developed, tested, and implemented Collabucate, a web-based tool for fostering social regulation of learning in collaborative learning. In this paper, we describe two cycles of a design-based research study in which Collabucate was implemented with Doctor of Pharmacy students. Collabucate was created according to Järvelä and colleagues' (2015) design principles for supporting socially shared regulation of learning and our design proposition to provide direct instruction of social regulation strategies. During two implementations, we collected and analyzed log data, student ratings, and focus groups. We improved design elements between the two design-based research cycles. Study results supported Järvelä and colleagues’ (2015) design principles and our design proposition. However, further work is required to improve how design elements are embodied in web-based tools. Based on our experience, we outline achievements, challenges, and future considerations for related tools and research. The study contributes to a growing number of tools created to foster awareness and effective regulation in collaborative learning.