As research increasingly relies on computational methods, the reliability of scientific results depends on the quality, reproducibility, and transparency of research software. Ensuring these qualities is critical for scientific integrity and discovery. This paper asks whether Research Software Science (RSS)–the empirical study of how research software is developed and used–should be considered a form of metascience, the science of science. Classification matters because it could affect recognition, funding, and integration of RSS into research improvement. We define metascience and RSS, compare their principles and objectives, and examine their overlaps. Arguments for classification highlight shared commitments to reproducibility, transparency, and empirical study of research processes. Arguments against portraying RSS as a specialized domain focused on a tool rather than the broader scientific enterprise. Our analysis finds RSS advances core goals of metascience, especially in computational reproducibility, and bridges technical, social, and cognitive aspects of research. Its classification depends on whether one adopts a broad definition of metascience–any empirical effort to improve science–or a narrow one focused on systemic and epistemological structures. We argue RSS is best understood as a distinct interdisciplinary domain that aligns with, and in some definitions fits within, metascience. Recognizing it as such can strengthen its role in improving reliability, justify funding, and elevate software development in research institutions. Regardless of classification, applying scientific rigor to research software ensures the tools of discovery meet the standards of the discoveries themselves.

The U.S. Department of Energy (DOE) Exascale Computing Project (ECP) funded the development of new (and the transformation of important existing) applications, libraries, and tools that realized improvement in performance and capabilities of often 100 times or more on emerging exascale computers. This exceptional gain inspired the title of this special issue: Transforming Science through Software: Improving while delivering 100X. The term 100X refers to advancing capabilities in modeling, simulation, and analysis by a factor of 100 or more using some combination of new algorithms, optimization techniques, software libraries, and programming models, coupled with the next generation of hardware for high-performance computing (HPC). The papers in this issue share experiences with the practice and science of scientific software development, with an emphasis on developing a coherent, portable, and sustainable HPC software ecosystem for next-generation computational science. We hope to foster expanded community efforts related to the fundamental role of sustainable scientific software ecosystems in advancing the computing sciences.

The Exascale Computing Project (ECP) was one of the largest open source scientific software development projects ever. It supported approximately 1000 staff from the U.S. Department of Energy laboratories, and university and industry partners. About 250 staff contributed to 70 scientific libraries and tools to support applications on multiple exascale computing systems that were also under development. Funded as a formal construction project, ECP was required to use earned-value management, based on milestones, and a key performance parameter system based, in part, on integrations. With accelerated delivery schedules and significant project risk, we also emphasized software quality using community policies, automated testing, and continuous integration. Software development kit teams provided cross-team collaboration, and products were delivered via E4S, a curated portfolio of libraries and tools. In this article, we discuss the organizational and management elements of ECP that enabled the delivery of libraries and tools, our lessons learned, and our next steps.

In this paper, we discuss the need for an integrated software stack that unites artificial intelligence (AI) and modeling and simulation (ModSim) tools to advance scientific discovery. The authors advocate for a unified AI/ModSim software ecosystem that ensures compatibility across a wide range of software on diverse high-performance computing systems, promoting ease of deployment, version management, and binary distribution. Key challenges highlighted include balancing the distinct needs of AI and ModSim, especially in terms of software build practices, dependency management, and compatibility. The document underscores the importance of continuous integration, community-driven stewardship, and collaboration with the Department of Energy (DOE) to develop a portable and cohesive scientific software ecosystem. Recommendations focus on supporting standardized environments through initiatives like the Extreme-scale Scientific Software Stack (E4S) and Spack to foster interdisciplinary innovation and facilitate new scientific advancements.

The U.S. Department of Energy (DOE) Exascale Computing Project (ECP) funded the development of new (and the transformation of important existing) applications, libraries, and tools that realized improvement in performance and capabilities of often 100 times or more on emerging exascale computers. This exceptional gain inspired the title of this special issue: Transforming Science through Software: Improving while delivering 100X. The term 100X refers to advancing capabilities in modeling, simulation, and analysis by a factor of 100 or more using some combination of new algorithms, optimization techniques, software libraries, and programming models, coupled with the next generation of hardware for high-performance computing (HPC). The papers in this issue share experiences with the practice and science of scientific software development, with an emphasis on developing a coherent, portable, and sustainable HPC software ecosystem for next-generation computational science. We hope to foster expanded community efforts related to the fundamental role of sustainable scientific software ecosystems in advancing the computing sciences.

The development of scientific software—a cornerstone of long-term collaboration and scientific progress—parallels the development of other types of software but still poses distinct challenges, especially in high-performance computing. Although web searches yield numerous resources on software engineering, there is still a scarcity specifically for scientific software development. This article introduces the Better Scientific Software site (https://bssw.io), a platform that hosts a community of researchers, developers, and practitioners who share their experiences and insights on scientific software development. Since 2017, this collaborative hub has gained traction within the scientific computing community, attracting a growing number of readers and contributors eager to share ideas and elevate their software development practices. In sharing the BSSw.io site’s story, we hope to encourage further growth of the BSSw.io community through both readership and contributors, with a long-term goal of fostering culture change by increasing emphasis on best practices in scientific software.

The Exascale Computing Project (ECP) Software Technology and Co-Design teams addressed the growing complexities in high-performance computing (HPC) by developing scalable software libraries and tools that leverage exascale system capabilities. As we enter the exascale era, the need for reusable, optimized software solutions that can handle the unique challenges posed by these systems becomes increasingly important. The primary challenges the ECP teams faced were to create software libraries and tools that are performant on exascale architectures and portable and usable across diverse hardware platforms. Efforts addressed issues related to concurrent execution, memory management, and the integration of heterogeneous computing resources, such as GPUs from multiple vendors. The ECP’s strategy involved a structured development process encompassing the creation, optimization, and deployment of software in collaboration with industry, academia, and national laboratories. The project was organized into several technical areas: co-design of domain-specific suites with target applications, programming models and runtimes, development tools, mathematical libraries, data and visualization tools, and software ecosystem and delivery mechanisms. ECP has successfully developed a large portfolio of software libraries and tools that demonstrate significant improvements in performance and scalability on exascale systems. These products have been integrated into the Department of Energy’s computing facilities, supporting various scientific applications and ensuring robust performance across different hardware setups. ECP advancements in software development for exascale computing highlight the importance of a collaborative and adaptive approach to handling next-generation HPC systems complexities. The lessons learned emphasize the need for continuous engagement with end-users and vendors, and the importance of maintaining a balance between innovation and practical implementation. Future efforts will focus on ensuring scalability, keeping pace with rapid hardware advancements, and further enhancing the interoperability and usability of the software ecosystem. Subsequent articles in this special issue provide in-depth discussions and case studies into specific library and tool efforts.

Computational and data-enabled science and engineering are revolutionizing advances throughout science and society, at all scales of computing. For example, teams in the U.S. DOE Exascale Computing Project have been tackling new frontiers in modeling, simulation, and analysis by exploiting unprecedented exascale computing capabilities-building an advanced software ecosystem that supports next-generation applications and addresses disruptive changes in computer architectures. However, concerns are growing about the productivity of the developers of scientific software, its sustainability, and the trustworthiness of the results that it produces. Members of the IDEAS project serve as catalysts to address these challenges through fostering software communities, incubating and curating methodologies and resources, and disseminating knowledge to advance developer productivity and software sustainability. This paper discusses how these synergistic activities are advancing scientific discovery-mitigating technical risks by building a firmer foundation for reproducible, sustainable science at all scales of computing, from laptops to clusters to exascale and beyond.

in FY20. It is clear from this assessment that performance of the linear solver will have to be a focus in FY19.

Increasingly powerful and affordable computing has revolutionized scientific and scholarly discovery across a broad range of fields. Computing relies on software, which has been rapidly growing in scope, diversity, and complexity. At the same time, the methods, processes, and tools used to produce and utilize this essential software are often ad hoc, and the study and improvement of them are often done without the benefit of direct funding or prioritization. Consequently, concerns are growing about the productivity of the developers and users of scientific software, its sustainability, and the trustworthiness of the results that it produces. Increased investment, especially in the characterization and improvement of how scientific software is developed and used, is important for sustaining and improving the impact of software as the scope and complexity of scientific efforts expand. Without this investment, we face the risk of diminishing returns on our software investments because the demands for increased functionality, usability, reliability, and more will not be sufficiently met. The US Department of Energy Office of Science (DOE/SC) is at the forefront of modern software-enabled scientific discovery across numerous areas of computational, experimental, and observational science, including major investments in national user facilities that support these activities. For many years, DOE/SC software investments have provided tremendous value to the scientific community. We want to continue and further improve the value of DOE/SC software efforts by using a scientific approach to understanding and improving how scientific software is developed and used. In December 2021, the DOE/SC Office of Advanced Scientific Computing Research (ASCR) convened a workshop on basic research needs for the Science of Scientific-Software Development and Use (SSSDU). Through keynote presentations, lightning talks, and breakout groups, which built on insights from 124 pre-workshop position papers, participants discussed the current practice of software development, maintenance, evolution, and use, and considered how the scientific method could be used to examine these practices and develop more evidence-based approaches to enhance the impact of software and computing on all areas of science. Workshop participants identified three priority research directions (PRDs) and three important crosscutting themes that center on the following overarching insight: Software has become an essential part of modern science, impacting discoveries, policy, and technological development. To maintain and improve confidence in science delivered via software, we must improve the processes and tools that help us create and use software, and this enhancement requires a deep understanding of the diverse array of teams and individuals doing the work.

Software plays a central role in scientific discovery. Improving how we develop and use software for research can have both broad and deep impacts on a spectrum of challenges and opportunities society faces today. The emergence of the research software engineer (RSE) role correlates with the growing complexity of scientific challenges and the diversity of software team skills. In this article, research software science (RSS), an idea related to RSE and particularly suited to research software teams, is described. RSS promotes the use of scientific methodologies to explore and establish broadly applicable knowledge. Using RSS, we can pursue sustainable, repeatable, and reproducible software improvements that positively impact research software toward improved scientific discovery.

The Collegeville Workshop Series on Scientific Software brings together members from the academic, industry, and research laboratory communities to discuss important and emerging topics in the development and use of software for scientific discovery. The 2021 workshopb focused on the role of teams. This special issue contains five articles selected from the 26 white papers and teatime submissions to the workshop. Scientific software development is primarily a team activity.

Reproducibility has a foundational role in ensuring robust and trustworthy research, but achieving reproducibility can be challenging. This theme issue explores these challenges along with research and implementations across communities addressing them, with the goal of understanding the impact of existing solutions and synthesizing lessons learned and emerging best practices.

Emerging exascale architectures and systems will provide a sizable increase in raw computing power for science. To ensure the full potential of these new and diverse architectures, as well as the longevity and sustainability of science applications, we need to embrace software ecosystems as first-class citizens.

Scientific software underpins a large and growing number of scientific discoveries and engineering advances. At the same time, it is a small portion of the total software produced, has little literature, or focus on improvement and community building relative to other, larger software domains. The Collegeville Workshop Series on Scientific Software provides a platform for scientific software teams and stakeholders to exchange information on the nature, challenges, and technical and cultural approaches to improve key elements of the scientific software enterprise. By soliciting white papers, posters, and tea-time themes, these workshops gather content to provide a three-day live event, plus recorded presentations and interviews that enable community members to learn from each other. The outcomes from the first two workshops, in 2019 and 2020, have provided a body of knowledge and experience that expands the size and recognition of software as an important part of the scientific enterprise. The 2021 workshop, focused on software teams, intends to further expand the conversation, and continue to build community around scientific software to improve its impact on science and engineering.