STEM education research has historically been under-equipped, relying on standardized tests and questionnaires while other fields deploy space telescopes and particle accelerators. What would happen if we designed research infrastructure for STEM education to address our priority needs? The INTERACT Incubator is a research coordination network whose goal was to develop a field-initiated vision of novel infrastructure that would enable aspirational research that could advance equity in STEM education. The incubator brought together a diverse cohort of experimental social psychologists who study social cues in STEM, experimental cognitive psychologists who study learning in authentic classroom settings, as well as education stakeholders and technologists with expertise in digital infrastructure for education. In Phase 1 we conducted a needs assessment, where we brainstormed aspirational research studies and identified three core infrastructure requirements: coordinated data collection and measurement systems, sustainable large-scale research-practice partnership frameworks, and knowledge repositories combined with professional learning networks. In Phase 2 we designed an integrated solution to address these needs. The INTERACT Incubator’s solution differs from existing research infrastructure because ours was systematically designed to address research needs identified by the field itself, rather than building research services on top of existing research capacities or operational systems. This commentary documents a consensus vision for novel infrastructure that would enable the research needed to achieve meaningful progress in STEM education.

This paper investigates the return on investment (ROI) in cyberinfrastructure (CI) facilities and services by comparing the value of end products created to the cost of operations. We assessed the cost of a US CI facility called XSEDE and the value of the end products created using this facility, categorizing end products according to the International Integrated Reporting Framework. The US federal government invested approximately 0.3B in operating the XSEDE ecosystem from 2016–2022. The estimated value of end products facilitated by XSEDE ranges from around4.7B to 22.7B or more. Credit for the majority of these end products is shared among various contributors, including the XSEDE ecosystem. Granting the XSEDE ecosystem a seemingly reasonable percentage of credit for its contributions to end product creation suggests that the return on federal investment in the XSEDE ecosystem, in terms of value of end products created, was greater than one and possibly far greater than one. The Framework proved useful for addressing this question. Earlier work showed that the value of services provided by XSEDE was significantly greater than the cost of those services to the US federal government—a positive return on investment for delivery of services. Analyzing the financial efficiency of operations and the financial value of end products are two means for assessing the success of CI facilities in financial terms. Financial analyses should be used as one of many approaches for evaluating the success of CI facilities.

The purpose of this study is to propose a framework through which an organization could identify, describe, and measure ten factors categorized as soft skills shown to undergird the thriving culture of XSEDE, NSF’s largest cyberinfrastructure project during 2011–2022. We conducted 54 interviews of XSEDE collaborators; these interviews proceeded in two phases, with the second functioning as member checks to ensure our findings accorded with respondents’ experiences. Based on a grounded theory analysis of these 54 semi-structured interviews with a range of XSEDE stakeholders, we identified 10 social and cultural factors contributing to XSEDE’s success: (1) communicating intentionally, (2) maintaining relationships, (3) helping colleagues and users, (4) trusting colleagues, (5) having thick skin, (6) assuming the best in others, (7) respecting colleagues, (8) remaining open-minded, (9) practicing transparency, and (10) showing appreciation. We believe these ten factors can be used to create a methodology for developing soft skills metrics for virtual organizations and cyberinfrastructure projects similar to XSEDE. This would allow stakeholders to proactively measure engagement in behaviors, practices, and mindsets conducive to generating a thriving culture for virtual collaborations.

For researchers seeking to improve education, a common goal is to identify teaching practices that have causal benefits in classroom settings. To test whether an instructional practice exerts a causal influence on an outcome measure, the most straightforward and compelling method is to conduct an experiment. While experimentation is common in laboratory studies of learning, experimentation is increasingly rare in classroom settings, and to date, researchers have argued it is prohibitively expensive and difficult to conduct experiments on education in situ. To address this challenge, we present Terracotta (Tool for Education Research with RAndomized COnTrolled TriAls), an open-source web application that integrates with a learning management system to provide a comprehensive experimental research platform within an online class site. Terracotta automates randomization, informed consent, experimental manipulation of different versions of learning activities, and export of de-identified research data. Here we describe these features, and the results of a live classroom demonstration study using Terracotta, a preregistered replication of McDaniel et al. (Journal of Applied Research in Memory and Cognition, 1(1), 18–26, 2012). Using Terracotta, we experimentally manipulated online review assignments so that consenting students alternated, on a weekly basis, between taking multiple-choice quizzes (retrieval practice) and reading answers to these quizzes (restudy). Students' performance on subsequent exams was significantly improved for items that had been in retrieval practice review assignments. This successful replication demonstrates that Terracotta can be used to experimentally manipulate consequential aspects of students’ experiences in education settings.

This paper uses accounting concepts—particularly the concept of Return on Investment (ROI)—to reveal the quantitative value of scientific research pertaining to a major US cyberinfrastructure project (XSEDE—the eXtreme Science and Engineering Discovery Environment). XSEDE provides operational and support services for advanced information technology systems, cloud systems, and supercomputers supporting non-classified US research, with an average budget for XSEDE of US$20M+ per year over the period studied (2014–2021). To assess the financial effectiveness of these services, we calculated a proxy for ROI, and converted quantitative measures of XSEDE service delivery into financial values using costs for service from the US marketplace. We calculated two estimates of ROI: a Conservative Estimate, functioning as a lower bound and using publicly available data for a lower valuation of XSEDE services; and a Best Available Estimate, functioning as a more accurate estimate, but using some unpublished valuation data. Using the largest dataset assembled for analysis of ROI for a cyberinfrastructure project, we found a Conservative Estimate of ROI of 1.87, and a Best Available Estimate of ROI of 3.24. Through accounting methods, we show that XSEDE services offer excellent value to the US government, that the services offered uniquely by XSEDE (that is, not otherwise available for purchase) were the most valuable to the facilitation of US research activities, and that accounting-based concepts hold great value for understanding the mechanisms of scientific research generally.

This paper explores the financial effectiveness of a national advanced computing support organization within the United States (US) called the eXtreme Science and Engineering Discovery Environment (XSEDE). XSEDE was funded by the National Science Foundation (NSF) in 2011 to manage delivery of advanced computing support to researchers in the US working on non-classified research. In this paper, we describe the methodologies employed to calculate the return on investment (ROI) for governmental expenditures on XSEDE and present a lower bound on the US government’s ROI for XSEDE from 2014 to 2020. For each year of the XSEDE project considered, XSEDE delivered measurable value to the US that exceeded the cost incurred by the Federal Government to fund XSEDE. That is, the US Federal Government’s ROI for XSEDE is at least 1 each year. Over the course of the study period, the ROI for XSEDE rose from 0.99 to 1.78. This increase was due partly to our ability to assign a value to more and more of XSEDE’s services over time and partly to the value of certain XSEDE services increasing over time. From 2014 to 2020, XSEDE offered an ROI of more than $1.5 in value for every $1.0 invested by the US Federal Government. Because our estimations were very conservative, this figure represents the lower bound of the value created by XSEDE. The most important part of ”returns” created by XSEDE are the actual outcomes it enables in terms of education, enabling new discoveries, and supporting the creation of new inventions that improve quality of life. In future work we will use newly developed accounting methodologies to begin assessing the value of the outcomes of XSEDE.

Ted Shawn’s all-male modern dance company, Ted Shawn and His Men Dancers, toured extensively throughout the United States from 1933 to 1939 with the explicit goal of making concert dance a legitimate career for men. Shawn trained his dancers and choreographed their performances with careful attention to theme and movement aesthetic, proffering a version of modern dance meant to counter prevailing cultural prohibitions against men dancing. One of the company’s early works, Labor Symphony (1934), depicts the evolution of work from field to factory while critiquing the equation of the male body with labor power within capitalist economic systems; it also presents itself as labor, countering modern preoccupations with bodies enervated through repetitive movement or sedentary work, and with alienation from community through overly individuated tasks. The dance exemplifies Shawn’s novel movement aesthetic while expanding notions of productive labor by placing work itself in the context of performance. Emphasizing its status as productive work for men, the four sections of Labor Symphony display the Men Dancers’ strength, agility, and musculature prominently, exposing the piece as both means and end: it produces both the dance itself and bodies capable of performing it, much as agricultural and manual labor offer both product and a body disciplined to produce it.

Building on the work of interdisciplinary literary and dance scholars from Frank Kermode to Carrie Preston, this paper attends to depictions of Isadora Duncan appearing in novels, poems, and portraits by modernist writers such as Max Eastman, John Dos Passos, and Gertrude Stein, among others. These diversely experimental texts hold in common a preoccupation with Duncan's perpetual motion, emphasizing shifts in her personal life, choreography, and performance quality over time. “Writing Bodies” theorizes the different expressive roles her image takes on, arguing that the texts use Duncan's image to query the relationship between embodied movement and its literary representations.