A multilevel-discrete time survival model may be appropriate for purely hierarchical data, but when data are non-purely hierarchical due to individual mobility across clusters, a cross-classified discrete time survival model may be necessary. The purpose of this research was to investigate the performance of a cross-classified discrete-time survival model and assess the impact of ignoring a cross-classified data structure on the model parameters of a conventional discrete-time survival model and a multilevel discrete-time survival model. A Monte Carlo simulation was used to examine the performance of three discrete-time survival models when individuals are mobile across clusters. Simulation factors included the value of the between-clusters variance, number of clusters, within-cluster sample size, Weibull scale parameter, and mobility rate. The results suggest that substantial relative parameter bias, unacceptable coverage of the 95% confidence intervals, and severely biased standard errors are possible for all model parameters when a discrete-time survival model is used that ignores the cross-classified data structure. The findings presented in this study are useful for methodologists and practitioners in educational research, public health, and other social sciences where discrete-time survival analysis is a common methodological technique for analyzing event-history data.

Background The Noyce Scholarship Program was created to attract and retain science, technology, engineering, and mathematics (STEM) teachers in high-need schools. Teacher support networks, and specifically mentorship support, have been linked to increased retention of high-quality teachers in the classroom. Using a sample of Noyce teachers, we used a multilevel model to explore how the characteristics and composition of novice teachers’ support networks are related to the likelihood that they receive mentorship support, and further, how characteristics common among Noyce programs are related to mentorship support. Results Findings suggest that the characteristics and composition of a teacher’s network, as well as certain Noyce program characteristics, contribute to the likelihood that teachers receive mentorship support from their larger support network. Implications The results of this study highlight the importance of considering how the design of teacher preparation programs may contribute to continued mentorship support for early career teachers, and ultimately, their retention in the classroom.

This innovative practice work in progress paper presents Biologically inspired design (BID) to transfer design principles identified in nature to human-centered design problems. The Biologically Inspired Design for Engineering Education (BIRDEE) program uses biologically inspired design to teach high school engineering in a way that uniquely engages students in the natural world. For high school students, identifying natural systems' analogues for human design problems can be challenging. Furthermore, it is often the case that students focus on and transfer superficial structures, rather than underlying design principles. Based on the Structure-Behavior-Function (SBF) design ontology, we developed a modified cognitive scaffold called Structure-Function-Mechanism (SFM) to assist students and teachers with identifying functionally similar biological analogies and identifying and transferring design principles. In this paper we describe SFM and its importance in BID and our observations from teaching SFM to high school teachers during a multi-week professional development workshop in the summer of 2020. Based on teachers' work artifacts, transcriptions of discussions, and focus groups, we highlight the challenges of teaching SFM and our plans to scaffold this important concept for students and teachers alike.

This innovative practice work in progress paper presents the Biologically Inspired Design for Engineering Education (BIRDEE) project, to create socially relevant, accessible, highly-contextualized biologically inspired design experiences that can be disseminated to high school audiences engineering audiences in Georgia and nationally. Curriculum units are 6–10 weeks in duration and will meet many standards for high school engineering courses in Georgia. There will be three curriculum units (one for each engineering course in the 3-course pathway), each building skills in engineering design and specific skills for BID. Currently in its second year, BIRDEE has developed its first unit of curriculum and has hosted its first professional development with 4 pilot teachers in the summer of 2020. The BIRDEE curriculum situates challenges within socially relevant contexts and provides cutting-edge biological scenarios to ignite creative and humanistic engineering experiences to 1) drive greater engagement in engineering, particularly among women, 2) improve student engineering skills, especially problem definition and ideation skills, and 3) increase students awareness of the connection and impacts between the engineered and living worlds. This paper describes the motivation for the BIRDEE project, the learning goals for the curriculum, and a description of the first unit. We provide reflections and feedback from teacher work and focus groups during our summer professional development and highlight the challenges associated with building BID competency across biology and engineering to equip teachers with the skills they need to teach the BIRDEE units. These lessons can be applied to teaching BID more broadly, as its multidisciplinary nature creates challenges (and opportunities) for teaching and learning engineering design.

Biologically inspired design has become increasingly common in graduate and undergraduate engineering programs, consistent with an expanding emphasis by professional engineering societies on cross-disciplinary critical thinking skills and adaptive and sustainable design. However, bio-inspired engineering is less common in K-12 education. In 2019, the NSF funded a K-12 project entitled Biologically Inspired Design for Engineering Education (BIRDEE), to create socially relevant, accessible, and highly contextualized high school engineering curricula focusing on bio-inspired design. Studies have shown that women and underrepresented minorities are drawn to curricula, courses, and instructional strategies that are integrated, emphasize systems thinking, and facilitate connection building across courses or disciplines. The BIRDEE project also seeks to interest high school girls in engineering by providing curricula that incorporate humanistic, bio-inspired engineering with a focus on sustainable and authentic design contexts. BIRDEE curricula integrate bio-inspired design into the engineering design process by leveraging design tools that facilitate the application of biological concepts to design challenges. This provides a conceptual framework enabling students to systematically define a design problem, resulting in better, more well-rounded problem specifications. The professional development (PD) for the participating teachers include six-week-long summer internships in university research laboratories focused on biology and bio-inspired design. The goal of these internships is to improve engineering teachers’ knowledge of bio-inspired design by partnering with cutting-edge engineers and scientists to study animal features and behaviors and their applications to engineering design. However, due to COVID-19 and research lab closures in the summer of 2020, the research team had to transfer the summer PD experience to an online setting. An asynchronous, quasi-facilitated online course was developed and delivered to teachers over six weeks. In this paper, we will discuss online pedagogical approaches to experiential learning, teaching bio-inspired design concepts, and the integration of these approaches in the engineering design process. Central to the online PD design and function of each course was the use of inquiry, experiential and highly-collaborative learning strategies. Preliminary results show that teachers appreciated the aspects of the summer PD that included exploration, such as during the “Found Object” activity, and the process of building a prototype. These activities represented experiential learning opportunities where teachers were able to learn by doing. It was noted throughout the focus group discussions that such opportunities were appreciated by participating teachers. Teachers indicated that the experiential learning components of the PD allowed them to do something outside of their comfort zone, inspired them to do research that they would not have done outside of this experience, and allowed them to “be in the student's seat and get hands-on application”. By participating in these experiential learning opportunities, teachers were also able to better understand how the BIRDEE curriculum may impact students’ learning in their classrooms.

Although teaching self-efficacy is associated with many benefits for teachers and students, little is known about how teachers develop a sense of efficacy in the early years of their careers. Drawing on survey ( N = 179) and interview ( N = 10) data, this study investigates the sources of self-efficacy in a national sample of teachers who participated in the Noyce program. All teachers completed an online survey that included both the Teacher Sense of Efficacy Instrument and open-ended items prompting them to reflect on the sources of their self-efficacy. Ten teachers participated in semi-structured follow-up interviews. Enactive mastery experiences were the most common source of self-efficacy identified by teachers, followed by social persuasions and vicarious experiences. Physiological and affective states were identified infrequently and more often related to negative experiences that lowered self-efficacy than to positive experiences. Beginning teachers identified more negative enactive experiences than either Novice (2–3 years experiences) or Career teachers. In interviews, teachers described how the sources combined or interacted to influence their self-efficacy. Findings contribute to better understandings of the sources of self-efficacy with implications for how best to support teachers at different stages of their careers.

In this case study we report on the use of a Next Generation Science Standards (NGSS)-aligned form of Structure-Behavior-Function, called Structure-Function-Mechanism (SFM), to teach four high school engineering teachers an approach for Biologically Inspired Design (BID). Functional theories of design describe a natural way in which designers solve design problems. They provide support for case-based and analogical-based reasoning systems and have been used successfully to teach BID to undergraduate students. We found that teachers instructed on BID practice and pedagogy using our modified theory were able to grasp the structural concepts and looked for clear markers separating mechanism (behavior) and function. Because of the systems-of-systems nature of most biological entities, these boundaries were often subjective, presenting unique challenge to teachers. As high school engineering teachers look for methods to enhance their pedagogy and to understand multidisciplinary content, these findings will inform future curriculum development and professional learning approaches for engineering education.

A three-level piecewise growth model (3L-PGM) can be used to break up nonlinear growth into multiple components, providing the opportunity to examine potential sources of variation in individual and contextual growth within different segments of the model. The conventional 3L-PGM assumes that the data are strictly hierarchical in nature, where measurement occasions (level 1) are nested within individuals (level 2) who are members of a single cluster (level 3). However, in longitudinal research, it is sometimes difficult for data structures to remain purely clustered during a study, such as when some students change classrooms or schools over time. One resulting data structure in this situation is known as a multiple membership structure, where some lower-level units are members of more than one higher-level unit. The new multiple membership PGM (MM-PGM) extends the 3L-PGM to handle multiple membership data structures frequently found in the social sciences. This study sought to examine the consequences of ignoring individual mobility across clusters when estimating a 3L-PGM in comparison to estimating a MM-PGM. MM-PGM estimates were less biased (especially in the cluster-level coefficient estimates), although we found substantial bias in cluster-level variance components across some conditions for both models.

In recent decades, invention education programs have been posited as a way to engage students in STEM through the hands-on process of designing their own inventions. The K-12 InVenture Prize (IP) program is an example of one such program, offering a platform for school-based invention education designed to be implemented during the academic year with Kindergarten through 12th-grade students. Previous literature has documented teachers’ perspectives on the positive student outcomes associated with participating in the K-12 IP program, as well as the challenges of implementing the K-12 IP curriculum during the school year. Based on these findings, the InVenture Prize Summer Accelerator was created with the purpose of exposing a greater number of K-12 students to invention education. A week-long summer program, modelled after the year-long K-12 IP curriculum, was designed and opened to rising 6th through 8th grade students. During the program, students were guided through the steps of the Design Thinking Process as they created their own inventions. The curriculum focused on developing students’ engineering knowledge, invention skills, and 21st century skills through team-based activities related to the steps in the Design Thinking Process, including empathizing, researching patents, prototyping, and pitching ideas. This paper presents findings from an evaluation of the 2019 InVenture Prize Summer Accelerator. This paper specifically addresses calls from the existing literature to document students’ self-reported outcomes of participating in InVenture programs, and is the first to describe the ways in which the K-12 IP program can be successfully adapted into a one-week summer program. Data were gathered through both qualitative and quantitative methods. Student outcomes and satisfaction were assessed using a preand post-survey design. Additionally, a focus group was conducted to investigate students’ perspective regarding specific activities and content included in the program. Pre-survey and focus group findings indicate that students were familiar with program content knowledge and 21st century skills prior to beginning the program. Despite this, there were increases in students’ confidence in program content knowledge, specifically knowledge related to the Design Thinking Process, and students’ 21st century skills, such as teamwork and communication. Additionally, students’ positive attitudes towards STEM and their intent to persist in STEM were maintained over the course of the week. Students were also satisfied with their Summer Accelerator experience and expressed an interest in continuing to invent after the program. These findings support the limited body of research on student outcomes associated with participation in InVenture programs, and offers unique insights into the outcomes associated with turning a school-based invention education program into a oneweek summer program in an out-of-school-time context.

Rule the Air! Summer Camp for High School StudentsDuring a weeklong summer camp, high school students were introduced to aviation and theairline industry. The camp curriculum centered on the use of airline simulation software toprovide students hands-on experience with some of the major challenges faced in the airlineindustry. The students formed teams that were tasked with creating and operating an airlinewhich included scheduling, staffing, fleet selection, maintenance, advertising, and ticket pricing.Throughout the week, lectures and activities were presented by graduate students and professorsto inform the decision-making process of the new airline executives. The topics coveredincluded: transportation engineering, the history of the airline industry, attributes that affectconsumer’s choice, reasons for passenger “no shows” and ways to predict the number ofpassengers that will not show up, game theory as applied to setting fares, scheduling techniques,the physics of flight, marketing programs, and frequent flyer incentives. The camp also includedfieldtrips to Atlanta’s Hartsfield Jackson International Airport and the Delta Heritage Museum.The curriculum culminated in a competition between the students’ airlines using the software tosimulate a real-world environment. The evaluation of this summer program utilized pre-postsurveys to measure the impact of camp experience on student self-efficacy and STEM interest.Furthermore, it was designed to determine students’ interest and understanding of the airlineindustry. The post-survey also consisted of open-ended questions focusing on students’ interestin pursuing a career in the airline industry after attending the camp. Results indicate that studentfamiliarity with the airline industry generally increased. Additionally, students generally feltmore confident performing activities related to the airline industry following the summerprogram. The majority of students (75%) also reported that they were interested in pursing amarketing or engineering degree in college. Overall, the evaluation results showed that the campwas a positive learning experience for students.

The role of measuring functional impairment holds an important place in research, clinical practice, and service provision for children and adolescents. Responding to the growing need to measure serious emotional disturbances at the local, state, and national level, the Columbia Impairment Scale (CIS) was developed in the early 1990s and has remained one of the several popular scales for assessing functional impairment. However, despite the growing popularity of the instrument in research and practice, only a few studies to date have specifically examined the psychometric properties of the CIS. In this article, we describe the results of the first item response theory analysis of the CIS utilizing nationally representative data from the Medical Expenditure Panel Survey ( N = 69,966). The results of our analysis lend support to the essential unidimensionality of the CIS and demonstrate that the scale is most reliable for those who exhibit high levels of functional impairment. Given the psychometric properties of the scale identified by our analysis, we contend that the CIS is a viable measure in the ongoing efforts to establish a national epidemiologic surveillance system to track the prevalence and impact of serious emotional disturbances in children and adolescents.

As interest increases in promoting STEM education in America, summer STEM programs are a promising option for increasing student engagement, interest, and knowledge of STEM. However, STEM programs pose challenges for evaluation, especially programs that serve a large number of students and address a wide range of STEM topics. This paper describes how a team of researchers and practitioners collaborated to design and implement an evaluation of a series of STEM summer programs held at a large, public university. The programs varied in the STEM topics they covered and the age of participants. This created challenges for evaluating a series of programs of such scope and variety. This paper will further describe the programs and the methods used to evaluate them. Illustrative results of the evaluation will be shared, in addition to lessons learned from our evaluation in the hopes that this paper can serve as a resource for those looking for a feasible way to evaluate large, diverse programs.

In the social sciences, applied researchers often face a statistical dilemma when multilevel data is structured such that lower-level units are not purely clustered within higher-level units. To aid applied researchers in appropriately analyzing such data structures, this study proposes a multiple membership growth curve model (MM-GCM). The MM-GCM offers some advantages to other similar modeling approaches, including greater flexibility in modeling the intercept at the time-point most desired for interpretation. A real longitudinal dataset from the field of education with a multiple membership structure, where some students changed schools over time, was used to demonstrate the application of the MM-GCM. Baseline and conditional MM-GCMs are presented, and parameter estimates were compared with two other common approaches to handling such data structures - the final school-GCM that ignores mobile students by only modeling the final school attended and the delete-GCM that deletes mobile students. Additionally, a simulation study was conducted to further assess the impact of ignoring mobility on parameter estimates. The results indicate that ignoring mobility results in substantial bias in model estimates, especially for cluster-level coefficients and variance components.

The program team operating an NSF Noyce Master Teacher program has been building a conceptual framework for developing teacher leaders. The program has focused its efforts on a group of 16 chemistry and physics teachers in Southeast high-needs schools. The conceptual framework is based on the view that teacher leaders are those individuals who retain a classroom presence, while simultaneously innovating practice and empowering others. A core principle of the framework is that embodying these attributes requires an ability to see oneself and the teaching practice in a way that goes beyond the expertise associated with content and pedagogical knowledge. Evidence drawn from years three and four of the NSF Noyce Master Teacher program are presented to demonstrate the participating teachers' understanding of the framework's components. These data also indicate the potential of the teachers to use the framework's principles to engage in leadership activity. Characterizing such understanding and the changes in it are foundational to determining the way such a framework influences teachers' approaches to leadership. This paper has implications for the growing number of teacher leader initiatives across the United States, and for the question of whether science, technology, engineering, and mathematics (STEM) teacher leadership should be considered separately from a general notion of teacher leadership.

The development of teacher leaders in science, technology, engineering, and mathematics has become a focus as demand grows on the national scale to improve student learning in these disciplines. As teachers' role in leadership continues to be redefined, research and professional development in teacher leadership will continue to evolve. Given the lack of a clear conceptualization of teacher leadership in the empirical literature, there is a clear methodological challenge for evaluators who are charged with assessing the impact of teacher leadership professional development programs. This paper describes how both the Utilization-Focused Evaluation and Theory-Driven Evaluation frameworks were used concurrently to design evaluation methods that were effective for assessing the impact of a dynamic teacher leadership program. The evaluation is specifically situated within the context of a Robert Noyce Scholarship Program, which aimed to grow veteran science teachers into teacher leaders. The paper describes how the evaluation frameworks used guided the evaluation methods, provides illustrative evaluation results, and states lessons learned from the author's experiences working within this context. This paper aims to provide an example of evaluation methods that could be replicated by evaluators' working within a Noyce or teacher leadership context.

During a weeklong summer camp, high school students were introduced to aviation and the airline industry. The camp curriculum used airline simulation software to provide students hands-on experience with some of the major challenges faced in the airline industry. The students formed teams that were tasked with creating and operating an airline, including purchasing aircraft; determining where and when the aircraft would fly while ensuring satisfactory maintenance checks; and determining staffing levels, advertising expenditures, and ticket prices. Throughout the week, lectures and activities presented by graduate students and professors aided the decision-making process of the new airline executives. The camp also included two field trips to Atlanta's Hartsfield-Jackson International Airport and the Delta Air Lines' Heritage Museum. The evaluation of this summer program utilized pre-post surveys to measure the impact of camp experience on student self-efficacy and STEM interest. The evaluation was also designed to determine students' interest and understanding of the airline industry. Overall, the evaluation results showed the camp was a positive learning experience for students.

This study describes the development, initial implementation, and preliminary findings from the evaluation of one school’s grant-funded effort to use mobile makerspaces to enhance project-based learning in STEAM (Science, Technology, Engineering, Arts, and Mathematics) disciplines. With the goal of integrating makerspace resources into its existing project-based learning (PBL) instructional model, the school has designed, built, and deployed 10 STEAM Trunks, each featuring materials and tools aligned to specific themes including “Arts & Crafts”, “Construction”, “Electronics”, and “3-D printing”. Following professional development sessions that introduced the STEAM Trunks and facilitated initial project planning, K-8 teachers began incorporating STEAM Trunks into PBL. Utilizing a transformative mixed-methods approach, this case study uses observation, survey, and document data to address the question, “to what extent and in what ways are mobile makerspaces being integrated into project-based learning at each grade level?” Observations were conducted as STEAM Trunks were utilized for projects planned by teachers at each grade level, both in classrooms during the regular school day and at special school events where students showcase their projects. Using a semi-structured observation protocol, researchers focus classroom observations on student engagement with STEAM Trunk materials in order to determine whether and in what ways making using the STEAM Trunks may foster the essential elements of project-based learning. An online survey administered at the end of each semester was used to gather teacher perspectives on the STEAM Trunk initiative. This survey asks K-8 teachers to share examples of how they have used of the STEAM Trunks, to reflect on whether and how the STEAM Trunks have enhanced project-based learning in their classroom, and to provide suggestions for additional materials or improvements to the STEAM Trunks. Finally, a variety of documents including photographs, meeting agendas and notes, implementation calendars, school websites, promotional materials, and communications (flyers, emails, etc.) were analyzed as a secondary data source detailing the STEAM Trunk development and implementation process. Preliminary results suggest both challenges and benefits related to STEAM Trunk implementation. Challenges include aligning STEAM Trunk materials with project-specific needs and logistics (scheduling, arranging secure and convenient storage, monitoring and replenishing STEAM Trunk supplies and equipment). Observation and survey data indicate a high level of adoption and satisfaction among K-8 teachers and increased teacher investment in project-based learning following STEAM Trunk deployment. Additionally, observations document numerous examples of students utilizing STEAM Trunk tools and materials to meaningfully engage in the engineering design process.

The InVenture Challenge is an innovation-driven experience for K-12 students that operates in partnership with an institute of higher education. Students across all grade levels and educational settings (e.g., regular, gifted, and AP classrooms in a variety of subject areas; after school programs) are eligible to participate. Generally, students work in small groups to develop an invention from problem-seeking to prototype over the course of multiple months. In the process, they present their ideas to others, solicit feedback, and iterate on their design multiple times. In the spring, students with the top inventions from their individual schools travel to Georgia Institute of Technology to pitch their inventions to judges and audience members in a statewide competition. The goal of this research is to understand the experiences of teachers and students within the program and the ways they benefit from participating. Initial research efforts have focused primarily on teachers’ experiences implementing the program. Through survey, focus group, and interview data collected over the past several years, teachers have also provided their perspectives about how the program has impacted their students. Across several academic years, teachers’ survey data reflects a high level of agreement that participation has had a positive impact on their students’ communication and teamwork skills, enthusiasm for learning about engineering and entrepreneurship, and knowledge of the engineering design process, how products are made, how to design a sales pitch, and how to start a business. In this paper, we summarize several years of teacher data related to perceived impact on students and present our first pre-post student survey data. This student survey data will allow us to directly investigate students’ experiences within the program and examine alignment with their teachers’ perceptions of student impacts. Together, this research will provide a multi-faceted view of invention education’s impacts on students.