Mathematical fluency is critical to upward mobility in many STEM professions, yet mathematics is often viewed as a gatekeeper that filters students out of STEM or hinders degree completion. Prior research has examined the mathematical self-efficacy and mathematical beliefs of K-12 students, and in some cases undergraduates. Here, we present and analyse data from five undergraduate STEM majors engaged in organising and leading K-12 outreach. Reflective journals and semi-structured interviews provided insights into their mathematical beliefs and mathematical self-efficacy. In all cases, even those students majoring in mathematics-rich disciplines, we observed gaps in mathematical self-efficacy, unfavourable views of mathematics, or both. Each undergraduate in our sample could reasonably be classified as 'successful', in that they were making steady progress through their degree programmes and were engaged in their studies. Additionally, their participation in this programme can be seen as a proxy for engagement in their pursuit of a STEM degree. Our observations therefore suggest that a deeper investigation is necessary of how engaged undergraduate STEM majors, including those moving steadily towards graduation, feel about mathematics and their abilities as mathematics learners. To that end, we conclude with a discussion of potential next steps and implications for mathematics educators.

The purpose of the present study was to perform a cross-validation of an existing measure of science students’ motivational traits using the Rasch modeling approach. The validity of the Self-determination, Purpose, Identity, and Engagement in Science (SPIRES) survey was originally investigated using factor analysis, but a secondary validation of this instrument has not yet been published. This is a recommended practice when using a psychometric instrument within a new context or with a different student population. In this validity study, we took a Rasch modeling approach instead of factor analysis because, unlike factor analysis, Rasch modeling is sample-independent. The original factor analysis validation of the SPIRES suggested the survey is composed of three larger ideas or constructs, while our Rasch modeling results suggest there are four constructs. Since our Rasch analyses were sample-independent, we conclude that the SPIRES survey is a four construct survey and may be treated as such across educational contexts without further need to validate using factor analysis, which could continuously produce inconsistent results. These results provide the basis for a validity argument for researchers using the SPIRES in their work. Our work also demonstrates an advantage to using a Rasch modeling approach over factor analysis for instrument validation.

Understanding the effectiveness of peer mentor training in online learning environments prepares students for their roles as online peer mentors. In the Learning Assistant peer mentor program, Learning Assistants (LAs) play a crucial role in supporting student learning and engagement in STEM courses. Due to the shift to online learning during the COVID-19 pandemic, many LAs began working in online synchronous and asynchronous courses. We gathered information on challenging interactions encountered by LAs, and the frequencies of these interactions were analyzed and compared between the two learning environments. The results revealed that while no new challenging interactions emerged in the online environment, there were nuances in how existing interactions manifested online. In this article, we present new challenging interaction scenarios designed for online learning environments. These scenarios aim to enhance the training in the LA pedagogy course. Furthermore, the study highlights the significance of behavioral engagement barriers, such as unprepared and disinterested students, which had higher frequencies in both in-person and online environments. These findings help create a foundation for online LA training by incorporating new scenarios and focusing on behavioral engagement barriers to help the LA pedagogy course better equip LAs to navigate challenging interactions and support student learning.

In this paper, we critically examine the way in which scholars have traditionally defined and problematized attrition. Through a series of examples of large-scale intervention impact studies, we share insights about the sources and consequences of attrition that expand our notion of how and why attrition occurs. We also discuss potential steps for anticipating, mitigating, and responding to attrition in the dynamic context of schooling. By expanding our understanding of attrition, we hope to engage the field in further dialogue that could lead to policies and practices that might lessen the potential impacts not only on our ability to conduct research, but also our ability to advance the learning of teachers and their students.

The American Chemical Society holds supporting diverse student populations engaging in chemistryas a core value. We analyzed chemical concept inventory scores from 4,612 students across 12institutions to determine what inequities in content knowledge existed before and after introductorycollege chemistry courses. We interpreted our findings from a Quantitative Critical (QuantCrit)perspective that framed inequities as educational debts that society owed students due to racism,sexism, or both. Results showed that society owed women and Black men large educational debtsbefore and after instruction. Society’s educational debts before instruction were large enough thatwomen and Black men’s average scores were lower than White men’s average pretest scores even afterinstruction. Society would have to provide opportunities equivalent to taking the course up to two anda half times to repay the largest educational debts. These findings show the scale of the inequities inthe science education systems and highlight the need for reallocating resources and opportunitiesthroughout the K-16 education system to mitigate, prevent, and repay society’s educational debts fromsexism and racism.

Education researchers often compare performance across race and gender on research-based assessments of physics knowledge to investigate the impacts of racism and sexism on physics student learning. These investigations' claims rely on research-based assessments providing reliable, unbiased measures of student knowledge across social identity groups. We used classical test theory and differential item functioning (DIF) analysis to examine whether the items on the Force Concept Inventory (FCI) provided unbiased data across social identifiers for race, gender, and their intersections. The data was accessed through the Learning About STEM Student Outcomes platform and included responses from 4,848 students posttests in 152 calculus-based introductory physics courses from 16 institutions. The results indicated that the majority of items (22) on the FCI were biased towards a group. These results point to the need for instrument validation to account for item bias and the identification or development of fair research-based assessments.

A growing part of the efforts to promote student engagement and success in undergraduate STEM are the family of Student Support and Outreach Programs (SSOPs), which task undergraduate students with providing support and mentoring to their peers and near-peers. Research has shown that these programs can provide a variety of benefits for the programs’ recipients, including increased academic achievement, satisfaction, retention, and entry into STEM careers. This paper extends this line of inquiry to investigate how participation in these programs impacts the undergraduate STEM students that provide the mentoring (defined here as undergraduate mentor-teachers or UMTs). We use activity theory to explore the nature of metacognition and identity development in UMTs engaged in two programs at a public urban-serving university in the western USA: a STEM Learning Assistant program and a program to organize middle and high school STEM clubs. Constructs of metacognition and identity development are seen as critical outcomes of experiential STEM inreach and outreach programs. Written reflections were collected throughout implementation of two experiential STEM inreach and outreach programs. A thematic analysis of the reflections revealed UMTs using metacognitive strategies including content reflection and reinforcement and goal setting for themselves and the students they were supporting. Participants also showed metacognitive awareness of the barriers and challenges related to their role in the program. In addition to these metacognitive processes, the UMTs developed their science identities by attaching different meanings to their role as a mentor in their respective programs and setting performance expectations for their roles. Performance expectations were contingent on pedagogical skills and the amount and type of content knowledge needed to effectively address student needs. The ability to meet students’ needs served to validate and verify UMTs’ role in the program, and ultimately their own science identities. Findings from this study suggest that metacognitive and identity developments are outcomes shaped not only by undergraduate students’ experiences, but also by their perceptions of what it means to learn and teach STEM. Experiential STEM inreach and outreach programs with structured opportunities for guided and open reflections can contribute to building participants’ metacognition and enhancing their science identities.

In the midst of the COVID-19 pandemic, institutions of higher education have made a rapid transition to teaching online. At the University of Colorado Denver, most of our lower division science courses are normally taught in a face-to-face modality. Some of our core biology, chemistry, physics, and mathematics courses are taught using Learning Assistants (LAs), who work as peer learning facilitators and help faculty transform the courses to be more student centered (Otero, 2006; Talbot et al., 2015). In these Learning Assistant supported courses, the move to online teaching and learning was supported by LAs. Data from faculty and LAs showed that the LAs shifted their role to support the transition, and that their support was valuable in the new online modality. Beyond their traditional roles, LAs’ presence in an online course may generate a stronger sense of community within the remote course, facilitate virtual communication between the instructor and students, promote participation in and moderate online course forums, and advise faculty about students’ technological barriers. As we move into the next academic year facing continued online teaching for many of these core courses, LAs will be integral in the design and support of online learning communities. REFERENCES Otero, V. (2006). The Colorado Learning Assistant Model: A multidisciplinary approach to teacher recruitment and preparation. Bulletin of the American Physical Society. Talbot, R. M., Hartley, L. M., Marzetta, K., & Wee, B. (2015). Transforming undergraduate science education with learning assistants: Student satisfaction in large enrollment courses. Journal of College Science Teaching, 44(5), 24–30.

Background The success of the learning assistant (LA) model has largely been attributed to LA facilitation of active learning tasks. A deeper understanding of how LAs facilitate these tasks would inform LA training and support successful adoption of the LA model. Our investigation of LA actions during their interaction with students in the classroom contributes to that understanding. We present and discuss the development of the action taxonomy for learning assistants (ATLAs), as well as illustrate its applicability by presenting some analyses that were conducted on sample data. Results The LAs carried out several different actions that we categorized broadly as LA-Directed Facilitation, LA-Guided Facilitation, Advice, Feedback, Course-Related Talk, and Non-Course-Related Talk. LA-Directed Facilitation and LA-Guided Facilitation were the most common types of actions observed. We found that LA actions varied by course. Conclusions ATLAs is a tool that can be used to examine LA actions. In our sample data set, LAs undertook many different actions during interactions with students which indicates that LAs play several different roles in the classroom. These findings have practical implications not only for faculty seeking to implement a peer instruction model such as the LA model, but also for instructors wanting to utilize LAs in their courses more effectively. Understanding what the LAs are doing during interactions with students can provide us insight into the different roles that LAs undertake. Knowledge of these roles will guide effective training, feedback, and direction of LAs, particularly during the pedagogy course.

Learning Assistants (LAs) help students develop a deeper understanding of content and are particularly effective during active learning instruction. A foundational pillar of the LA model is the LA pedagogy course, which teaches LAs about evidence-based instruction and about how students learn (Otero et al., 2010). From LA survey responses, this study identifies interactions between LAs and students that have the potential to negatively impact the classroom environment and how other students learn—what we call "challenging interactions." Challenging interaction training was developed for LAs taking the pedagogy course by using scenarios that LAs can act out and reflect on in class. This training aims to guide LAs as they develop their own strategies for how to properly navigate these interactions. Because of the potential negative impacts of these interactions, training LAs to address and manage these situations is important. If LAs can properly navigate these challenging interactions, they will be better able to facilitate deeper learning in their respective LA-supported classrooms. Additional informationNotes on contributorsAlicia PurtellAlicia Purtell is an undergraduate in the Department of Psychology and Neuroscience at Baylor University in Waco, Texas.Robert TalbotRobert Talbot is an associate professor of science education at the University of Colorado Denver.Michael E. MooreMichael E. Moore(michael.edward.moore@gmail.com) is a postdoctoral research associate at the University of Nebraska-Lincoln.

There is strong evidence that the implementation of active learning in undergraduate science courses can lead to increased student conceptual understanding and course achievement, but we still do not know what specific characteristics of active learning contribute the most to student success. Our work examines the tasks that students are asked to engage with during active learning, with the goal of investigating the relationship between different task characteristics and student level outcomes. To this end, we are working to characterize the active learning tasks that students engage with in the classroom with respect to authenticity and cognitive depth. This paper presents our characterization of the tasks we have collected from four introductory physics courses at three institutions and discusses the relationship between these characteristics and student gains on the Force and Motion Conceptual Evaluation instrument.

Measuring student learning is a complicated but necessary task for understanding the effectiveness of instruction and issues of equity in college STEM courses. Our investigation focused on the implications on claims about student learning that result from choosing between one of two commonly used methods for analyzing shifts in concept inventories. The methods are: Hake's gain (g), which is the most common method used in physics education research and other discipline based education research fields, and Cohen's d, which is broadly used in education research and many other fields. Data for the analyses came from the Learning Assistant Supported Student Outcomes (LASSO) database and included test scores from 4,551 students on physics, chemistry, biology, and math concept inventories from 89 courses at 17 institutions from across the United States. We compared the two methods across all of the concept inventories. The results showed that the two methods led to different inferences about student learning and equity due to g being biased in favor of high pretest populations. Recommendations for the analysis and reporting of findings on student learning data are included.

Asking questions is an important component of promoting inquiry and argumentation in the science classroom. We investigated the relationship between the cognitive depth and format of teacher-generated questions to be used with classroom response technologies. Twelve middle school science teachers were randomly assigned to write constrained or free-response questions on four different topics and to rate the cognitive depth of those questions. Using Bloom’s Taxonomy to guide question classification, we found that the teacher-generated free-response questions were 5.58 times more likely to be at the understanding level than the remembering level and 2.05 time more likely to be at the applying level than the understanding level. Our findings provide evidence of a potential barrier to adopting inquiry-based science teaching practices.

The need for a more robust, well-trained STEM workforce is becoming increasingly acute in the U.S., and there is a clear need to recruit and retain a larger and more diverse population of undergraduate STEM majors. While numerous efforts to improve engagement and support in the traditional P-16 classroom have been implemented successfully, it is also critical to explore other types of activities that have potential for high impact. The STEM Club Leadership for Undergraduate STEM Education, Recruiting and Success (STEM-CLUSTERS) project at our large public research university in the Mountain West presents an outreach model to engage undergraduate STEM majors in developing and facilitating activities in local middle and high school STEM clubs. Through case studies, built upon data from reflective journals and semi-structured interviews, the project has identified a number of benefits to the first cohort of participants, which is comprised of eleven undergraduate students operating in interdisciplinary teams across five schools. In this paper we describe the essential elements of our outreach model and suggest benefits related to undergraduates’ content knowledge, metacognition, communication skills, and identity as a future STEM professional.

Score reliability is necessary for establishing a validity argument for an instrument, and is therefore highly important to investigate. Depending on the proposed instrument use and score interpretations, differing degrees of precision in measurement or reliability are required. Researchers sometimes fail to take a critical stance when investigating this important measurement property, and default to accepted values of commonly known measures. This study takes a multi-faceted approach to scrutinizing score reliability from a measure of STEM teacher strategic knowledge using rater agreement, classical test theory conceptions of reliability, and Generalizability Theory. This detailed examination provides insight about where the greatest gains in score reliability can be realized, given the design of the instrument and the context of measurement.

There is a clear need for valid and reliable instrumentation that measures teacher knowledge. However, the process of investigating and making a case for instrument validity is not a simple undertaking; rather, it is a complex endeavor. This paper presents the empirical case of one aspect of such an instrument validation effort. The particular instrument under scrutiny was developed in order to determine the effect of a teacher education program on novice science and mathematics teachers’ strategic knowledge (SK). The relationship between novice science and mathematics teachers’ SK as measured by a survey and their SK as inferred from observations of practice using a widely used observation protocol is the subject of this paper. Moderate correlations between parts of the observation-based construct and the SK construct were observed. However, the main finding of this work is that the context in which the measurement is made (in situ observations vs. ex situ survey) is an essential factor in establishing the validity of the measurement itself.

Student success in large enrollment undergraduate science courses which utilize "active learning" and Learning Assistant (LA) support is a complex phenomenon. It is often ill-defined, is likely impacted by many factors, and regularly interacts with a variety of treatments or interventions. Defining, measuring, and modeling student success as a factor of multiple inputs is the focus of our work. Because this endeavor is complex and multifaceted, there is a need for strong theoretical framing. Without such explicit framing, we argue that our findings would be uninterpretable. In this paper we describe our efforts to define that theoretical framework, present the framework, and describe how it defines our methodological approach, analyses, and future work.

Large enrollment undergraduate science courses are often seen as “gatekeepers” and tend to support less-than-ideal pedagogical approaches. Student satisfaction with teaching and learning and gains in student conceptual understanding in these courses is often limited at best. At University of Colorado Denver, the Learning Assistant (LA) Program supports the transformation of these large-enrollment science courses to include more interactive teaching strategies and learning opportunities. We find that students in these LA-supported courses are satisfied with these courses in part because of their use of LAs, primarily during the lecture meeting time. Students do not report using LA support as much outside of course lecture meetings. Further, students in an LA-supported General Biology course also exhibited much larger gains in conceptual understanding. We suggest that future work should investigate cross-group comparisons of cognitive and affective gains by factors such as ethnicity; class; gender; and interactions among students, LAs, and faculty. Transforming Undergraduate Science Education With Learning Assistants: Student Satisfaction in Large Enrollment Courses