The present investigation adopts an embodied cognition perspective to characterize students' learning interactions between gesture and haptic experience. Haptic technologies have emerged as a promising approach to science learning because they can simulate physical forces that students would otherwise not experience. Student-generated gestures complement haptic technology by offering insight into students' embodied understandings of physical forces. In this study, we examined the role of tangible haptic molecular models by analyzing students' spontaneous gestures and speech after their learning experiences with these models. We designed three learning conditions-Individual Haptic Model, Aggregate Haptic Models, and Equation-Based Instruction-to prepare undergraduate students for learning from an agent-based computer simulation of dynamic equilibrium. Guided by Knowledge in Pieces (KiP) theory, we employed a think-aloud protocol to investigate how haptic and computational environments cue students' knowledge resources and spontaneous gestures. Our analysis of students' speech and gesture revealed that the haptic experiences-particularly individual-level haptic-augmented the computational environment by fostering shifts in students' reasoning (1) from the phenomenological primitive equilibrium to balance and (2) from representing aggregate patterns to individual interactions. We discuss the implications of using gesture as a window into students' developing reasoning, as well as the role of haptic technology in designing embodied learning environments.

The present investigation adopts an embodied cognition perspective to characterize students' learning interactions between gesture and haptic experience. Haptic technologies have emerged as a promising approach to science learning because they can simulate physical forces that students would otherwise not experience. Student-generated gestures complement haptic technology by offering insight into students' embodied understandings of physical forces. In this study, we examined the role of tangible haptic molecular models by analyzing students' spontaneous gestures and speech after their learning experiences with these models. We designed three learning conditions—Individual Haptic Model, Aggregate Haptic Models, and Equation-Based Instruction—to prepare undergraduate students for learning from an agent-based computer simulation of dynamic equilibrium. Guided by Knowledge in Pieces (KiP) theory, we employed a think-aloud protocol to investigate how haptic and computational environments cue students' knowledge resources and spontaneous gestures. Our analysis of students' speech and gesture revealed that the haptic experiences—particularly individual-level haptic—augmented the computational environment by fostering shifts in students' reasoning (1) from the phenomenological primitive equilibrium to balance and (2) from representing aggregate patterns to individual interactions. We discuss the implications of using gesture as a window into students' developing reasoning, as well as the role of haptic technology in designing embodied learning environments.

Background: The embodied cognition perspective is a collection of theories that all explicate a fundamental relationship between the perceptions, actions, and movement of the body and how humans think and reason. The application of the embodied cognition perspective to instructional interventions—often referred to as embodied learning design—has the potential to aid computing education researchers and practitioners who want to meaningfully integrate more sensory and “hands on” activities in their instruction. Purpose: The objective of this paper is to describe a process for, and to model, how computing designers and instructors can interrogate the embodied affordances of pedagogical activities in computing. Analytic Approach: We describe 12 embodied affordances present across six pedagogical activities. Our goal is to model the analysis of pedagogical activities for their embodied affordances. For coherence, we selected six pedagogical activities related to arrays in Java. Implications: Embodied learning offers theoretical ideas and a process for interrogating the characteristics of visual and physical resources in computing education. As with any design process, iteration and assessment are necessary to ensure that an embodied learning intervention is effective, but previous research suggests that learning is increased when students’ perceptual and physical interactions align with learning goals.

Significant research has been conducted on how students’ gestures aid in learning scientific concepts, yet there remains a gap in understanding the impact of gesture-based interactions between students and simulations on their interpretation of visualized scientific phenomena. Addressing this, our paper presents a usability test conducted on a dynamic equilibrium visualization simulation developed for introductory college courses. Through a user study involving 40 participants, we conducted a qualitative evaluation to determine how students interpret gesture-controlled simulations. The findings confirm that students generally interpret visualized scientific concepts effectively and that interacting through gestures enhances their interpretation of the simulations. Additionally, this paper discusses the limitations of the current study and suggests directions for future research.

Immersive technologies have the potential to play unique roles in the collaborative problem-solving process. To advance research in these contexts, new methods of documenting group collaboration with technology are necessary. The goal of this paper is to develop and utilize a coding scheme that enables investigation of the nature of collaborative interactions within the context of a shared virtual reality (VR) and tablet computer astronomy simulation. This multi-device environment was integrated into the laboratory learning of an undergraduate introductory astronomy course. Two exemplar groups were identified and reviewed for diversity of collaboration and technology interactions. Informed by existing coding practices for collaborative learning, these observations were then used to generate the coding scheme. Coding results reveal that task complexity contributed to an increase in diverse collaboration, and users who switched between technology platforms showed marked differences in their interactions with group members. Moments of discourse when group members were sharing the same simulation view were typically oriented around the acquisition of new knowledge and were often followed by bursts of productive discourse. We discuss the utility of our coding scheme alongside implications of this work via insight into what constitutes understanding in collaborative contexts augmented by immersive and networked technologies.

Embodied learning is a nascent term drawing on theories emphasising the role of the body, and body-based interaction in knowledge creation. Whilst embodied learning research has articulated pedagogical and design implications which resonate with museum practice and emphasis on hands-on approaches, translation between embodied learning research and everyday practice is limited. A key challenge is conducting research that is both methodologically rigorous whilst providing tractable implications for complex practice contexts. Whilst this tension is endemic in educational research, the field of embodied learning presents unique challenges. Here, we draw on experiences from a four-year, multisite, academic-practitioner research project investigating embodied learning with young children (3–6 years) in science centres/museums to synthesise, illustrate, and critically reflect on four key challenges: theoretical framing (how embodied learning is conceptualised), nature of the experience (what makes it embodied), evaluating embodied learning, and logistical challenges (capturing multiple modes of interaction, social context, communication). These challenges are illustrated through case studies, contributing a methodological lens for both academics and practitioners investigating the role and implications of embodied learning in museums.

This chapter describes the theories, design, and analysis applied to a pair of collaborative mixed-reality simulations addressing different scientific content but linked by the crosscutting concept of “rates of change”. Body tracking technologies allowed users to interact with these simulations using one of two conventions: through gestures chosen by participants, or a prescribed gesture representing the slope of a line using one’s arms. The two simulations explore the relationship between inputs and outputs in the context of population ecology and climate change systems. Participant pairs were randomly assigned to one of 4 treatment groups for gestural convention (learner-defined or prescribed) and simulation order (population ecology or climate change first). Knowledge and engagement surveys found a significant overall increase in individual knowledge, but no significant difference across the four treatments. This motivated deeper analysis of the dialog and actions of the participant pairs as they engaged with the system, where it was found that the order of simulation exposure produced a shift in the ratio between collaborative directives and simulation behavior dialog. In addition, the gestures recorded in the learner-defined gesture group were also leveraged in un-expected ways, such as in facilitating synchronization between partners.

Extended reality technologies such as headset-based augmented reality (AR) unlock unique opportunities to integrate gestures into the collaborative problem-solving process. The following qualitative study documents the collection and analysis of group interaction data in an astronomy sky simulation across AR and tablet technologies in a classroom setting. A total of 15 groups were coded for episodes of on-task problem-solving, conceptual engagement, and use of gesture. Analysis of coded interactions assisted in identifying vignettes facilitating exploration, orientation, perspective sharing, and communication of mental models. In addition, the use of gesture by some groups enabled the creation of shared situated conceptual spaces, bridging the AR and tablet experiences and facilitating collaborative exchange of spatial information. The patterns of gesture and collaborative knowledge interactions documented here have implications for the design of future collaborative learning environments leveraging extended reality technologies.

There is growing interest in the educational applications of immersive virtual reality (VR) and augmented reality (AR).However, there have been scarce empirical studies that directly compare the learning affordances of AR and VR.This study examines how AR and VR versions of the same astronomy simulation affected learning and collaborative behavior in undergraduate astronomy classrooms.Results showed that both environments were effective for supporting student understanding of key astronomy ideas and problem-solving strategies (e.g., determining one's latitude and longitude).Students who used AR had stronger learning gains on the more challenging assessment items compared to the groups of students who used VR.Additionally, students who used AR made explicit attempts to reconcile their viewpoint with their group members' viewpoints through gesture and referencing each other's perspectives, whereas VR users relied more on verbal coordination.

Informing teacher education programs and impacting teaching practices has long been a stated goal of learning sciences research.As a first step in understanding the reality of this impact, we conducted a survey of individuals who identify as learning scientists from 39 different higher education institutions distributed internationally.Survey questions asked these learning scientists to describe both current engagement with teacher education programs at their institution and aspirations for future engagement.Results indicate that learning scientists see their current overall impact on teacher education as low-to-moderate, but they also perceive a high potential for stronger impact in the future.Barriers to successful engagement and specific ideas for how to build bridges with teacher education programs are detailed.

: Headset-based augmented reality (AR) unlocks unique opportunities to integrate gestures into collaborative problem-solving activities. This paper documents a collaborative astronomy sky simulation across AR and tablet technologies. Two groups of students were identified from a larger data corpus based on the amount of interactions within the AR headset. These groups were coded for episodes of on-task problem solving and instances of collaboration involving gestures. Gesture interaction analysis assisted in identifying collections of interactions facilitating exploration, orientation, perspective-sharing and communication of mental models. These patterns of interactions suggest productive use of AR technologies to collaborate and problem solve through the use of gestures.

The metaverse entails modes of interactivity that hold enormous potential for education in various contexts and domains. The metaverse represents a convergence of technologies, such as immersive virtual reality (VR) and augmented reality, that allow for multimodal engagements with digital objects, virtual environments, and people. In this article, we focus specifically on the ways by which VR interactions with the metaverse can enhance learning and education. In our view, two primary issues have so far hindered the development of successful metaverse educational applications: first, the lack of theory-driven designs that use technologies such as VR in ways that are consistent with what we know about how people learn, and second, insufficient methods of evaluating metaverse technologies that go beyond usability and instead capture their efficacy for improving learning outcomes. To address these issues, this article aims to explore how three learning theories—experiential learning theory, distributed cognition theory, and embodied learning theory—can be applied to the design of educational VR, and how these theories can be leveraged to better support educational applications of the metaverse. We also introduce two science education VR environments developed in our lab that employ these theories to demonstrate how the design and development of the educational metaverse can be guided by research and evaluated on its ability to generate new learning.

Science students frequently struggle to apply crosscutting concepts such as scale or rates of change, and do not always effectively differentiate between linear and nonlinear processes. We approach this challenge from an embodied cognition perspective, which suggests learning can be facilitated by engaging students in physical activities that are aligned with target concepts. This article describes two experiments where participants learned about exponential scales applied to measuring earthquakes (Richter scale) and the strength of acids/bases (pH scale). Whether participants engaged with an embodied simulation or used traditional instructional media for none, one, or both of these topics was manipulated across the two experiments. Experiment 1 recruited high school-aged participants and Experiment 2 recruited non-STEM undergraduate majors. Results from the experiments showed that using the embodied simulation led to greater declarative knowledge gains for the earthquake topic but not for acids/bases. Learning gains for the crosscutting concept (exponential growth) were higher for the embodied simulation participants on both topics. Especially notable was the finding that learning the crosscutting concept from an embodied simulation in one domain transferred to a second domain where the same concept was relevant. We discuss implications for applications of embodied cognition to the design of learning environments and to interventions that support learning transfer. Educational Impact and Implications Statement This research lends support to the notion that students' natural movements (e.g., hand or full-body gestures) can be leveraged to facilitate both the learning of core STEM concepts and the transfer of those ideas to new topics. The two experiments described in this article provide empirical support for embodied learning, as well as design guidance for those interested in using immersive learning technologies such as mixed reality to create impactful educational activities.

This paper describes a framework for making explicit the design decisions in the development of immersive and interactive STEM learning technologies. This framework consists of three components: (1) visual viewpoint, the location from which a visual simulation depicts observable components; (2) embodied interaction, the ways in which a learner can physically engage with the simulation interface; and (3) learners' roles, the purpose and the participation structure the technology presents to the learner. The recent literature on the design of STEM learning technologies is reviewed with the lens of how the three components have been leveraged and what, if any, rationale is provided for the design decisions that were made. The definition and review of each component is followed by a set of reflective questions intended to prompt researchers and designers to be more explicit about these decisions and the ways they are intended to impact student learning in both the design process and the reporting of their work. The paper concludes with a discussion of how the three components interact, and how their articulation can support theory building as well as the proliferation of more effective STEM learning technology designs.

Constructing causal mechanistic explanations of observable phenomena is a key science practice that is often challenging for students as most mechanisms involve interactions of unobservable entities and activities. In this study, we examined how gesturing with a computer simulation that depicts the molecular mechanism of thermal conduction supported middle-school students in constructing causal mechanistic explanations. We designed a gesture-augmented computer simulation in which students were cued to use hand gestures to control the simulation. These cued gestures represent core causal interactions of conduction and they prompt students to physically engage with the simulation in conceptually meaningful ways. In this study, we examined how 21 students used the simulation and explained thermal conduction in a semi-structured interview, followed by a mixed-methods analysis. Quantitative analysis shows that students moved toward articulating the canonical causal mechanistic explanation of thermal conduction using the simulation. Three representative cases were identified to explore how students' explanations were facilitated by cued gestures. The analysis shows two main ways the cued gestures supported all students in the study: (a) by helping them attribute causal agency to molecules rather than an entity called Heat, and (b) by reifying the core mechanism of molecular collisions in conduction. Furthermore, the case studies show how each student's unique ways of sensemaking impacted their gesture use. Implications for instruction with gestures and design of augmented environments are discussed.

Recent research has emphasized the importance of leveraging embodied interactions for learning critical STEM concepts. ELASTIC(3)S-an embodied environment for learning about cross-cutting concepts (i.e., non-linear growth)-allows learners to interact with different science simulations through whole-body gestures. Technological advances in gesture recognition can track and respond to students' gestures, however, there has been little investigation into how the gestures performed in these environments relate to subsequent learning. The need for sequential pattern recognition methods is critical in embodied learning if we are to understand how gestural interaction with a simulation facilitates learning. Using data collected via Microsoft Kinect V2 from twelve college students, we applied multivariate Dynamic Time Warping for clustering to identify gestural patterns in ELASTIC(3)S as evidence for embodied learning processes. Our findings showed that identified trends of simulation use were indicative of students' struggles to understand the underlying ideas or use of the system and were associated with learning performance. These indicators can potentially be used to leverage real time, in-simulation assistance and promote a more adaptive learning experience via embodied simulations.