Low-temperature kinetic plasma simulations using particle-in-cell (PIC) and Monte Carlo methods (DSMC/MCC) for the chemistry can provide many advantages over the more popular fluid simulations, including detailed information about the ion energy and angular distribution functions that are critical for plasma processing. In this presentation, two different types of simulations illustrating the power of kinetic modeling are demonstrated. The first is a macroscopic-scale simulation of an inductively coupled plasma (ICP). We demonstrate how implicit methods can make these challenging simulations feasible, and show that our numerical model captures salient physical effects (inductive coupling, sheath formation, plasma generation, etc.) of the ICP discharge. Efforts to hasten the convergence of these simulations to steady-state and to improve their predictive capabilities are also summarized. Secondly, we outline ongoing work to develop microscopic feature-scale simulations of a through-silicon-via etch process, obtaining potential boundary conditions and incident particle fluxes within the feature from larger-scale kinetic sheath computations. For both simulation types, ion energy-angle distributions at the wafer surface, electron kinetics, and the detailed physics of the sheath and presheath can be computed by our numerical model.

Kinetic equilibrium reconstructions make use of profile information such as particle density and temperature measurements in addition to magnetics data to compute a self-consistent equilibrium. They are used in a multitude of physics-based modeling. This work develops a multi-layer perceptron (MLP) neural network (NN) model as a surrogate for kinetic Equilibrium Fitting (EFITs) and trains on the 2019 DIII-D discharge campaign database of kinetic equilibrium reconstructions. We investigate the impact of including various diagnostic data and machine actuator controls as input into the NN. When giving various categories of data as input into NN models that have been trained using those same categories of data, the predictions on multiple equilibrium reconstruction solutions (poloidal magnetic flux, global scalars, pressure profile, current profile) are highly accurate. When comparing different models with different diagnostics as input, the magnetics-only model outputs accurate kinetic profiles and the inclusion of additional data does not significantly impact the accuracy. When the NN is tasked with inferring only a single target such as the EFIT pressure profile or EFIT current profile, we see a large increase in the accuracy of the prediction of the kinetic profiles as more data is included. These results indicate that certain MLP NN configurations can be reasonably robust to different burning-plasma-relevant diagnostics depending on the accuracy requirements for equilibrium reconstruction tasks.

Magnetic fusion experiments have the largest gradients in the world that are in steady state. These gradients lead to instabilities, and thus, most of plasma theory over the past four decades have been devoted to the development of instability theory and their use in interpreting plasma results. Instabilities in tokamaks lead to 3 different phenomenological outcomes: coherent saturation, turbulent saturation, and sudden relaxation (which may include disruptions). For long-wavelength instabilities, the NIMROD extended MHD code has emerged as an important tool for understanding tokamak instabilities. Because of the long history of NIMROD, a new version of it has been started at Tech-X to be able to address multi-species capabilities as well as exploit modern GPU systems. The new version builds on the previous version and improves the workflow by enabling new equilibria from experiments to be resolved. The fundamental formulation of the equations underlying NIMROD has also been explored in multiples ways. First, understanding instabilities in tokamaks through numerical methods is aided by being able to understand the rich history of analytic studies. A paper was published which aid in understanding the literature by simplifying some of the analytic machinery inherent in these studies. Extended MHD, like gyrokinetics, are quasineutral models. From a theoretical point of view, plasma quasineutrality approximation is best understood as the same as the magneto-quasistatic approximation of the Maxwell equations. Finally, the standard model of tokamak theory is that of instability theory. A simple dynamical systems model has been developed to better illustrate the strengths and weaknesses of this model.

The Portable Extensible Toolkit for Scientific Computation (PETSc) library provides scalable solvers for nonlinear time-dependent differential and algebraic equations and for numerical optimization; it is used in dozens of scientific fields, and has been an important building block for many computational geoscience applications. Starting from the terascale era in the 1990s and continuing into the present dawn of the exascale era, a major goal of PETSc development has been achieving the scalability required to fully utilize leadership-class supercomputers. We will describe some of the algorithmic developments made during the era in which achieving inter-node scalability was the primary challenge to enabling extreme-scale computation, and then survey the challenges posed by the current era in which harnessing the abundant fine-scale parallelism within compute nodes -- primarily in the form of GPU-based accelerators -- has assumed at least equal importance. We will discuss how the PETSc design for performance portability addresses these challenges while stressing flexibility and extensibility by separating the programming model used by application code from that used by the library. Additionally, we will discuss recent developments in PETSc's communication module, PetscSF, that enable flexibility and scalable performance across large GPU-based systems while overcoming some of the difficulties posed by working directly with the Message Passing Interface (MPI) on such systems. A particular goal of this talk will be to go beyond simply describing the work performed to prepare PETSc and simulation codes that rely on it to run on exascale-class systems, but to enumerate the challenges we encountered and to share the essential lessons learned that can help other developers to prepare and optimize their high-performance scientific computing codes for the exascale era.

This paper presents the steps followed to GPU-offload parts of the core solver of EFIT-AI, an equilibrium reconstruction code suitable for tokamak experiments and burning plasmas. For this work, we will focus on the fitting procedure that consists of a Grad–Shafranov (GS) equation inverse solver that calculates equilibrium reconstructions on a grid. We will show profiling results of the original code (CPU-baseline), as well as the directives used to GPU-offload the most time-consuming function, initially to compare OpenACC and OpenMP on NVIDIA and AMD GPUs and later on to assess OpenMP performance portability on NVIDIA, AMD and Intel GPUs. We will make a performance comparison for different spatial grid sizes and show the speedup achieved on NVIDIA A100 (Perlmutter-NERSC), AMD MI250X (Frontier-OLCF) and Intel PVC GPUs (Sunspot-ALCF). Finally, we will draw some conclusions and recommendations to achieve high-performance portability for an equilibrium reconstruction code on the new HPC architectures.

The communities that develop and support open-source scientific software packages are crucial to the utility and success of such packages. Moreover, they form an important part of the human infrastructure that enables scientific progress. This article discusses aspects of the Portable Extensible Toolkit for Scientific Computation community, its organization, and technical approaches that enable community members to help each other efficiently and effectively.

Hard and soft infrastructure work in tandem to accomplish a particular enterprise. Soft infrastructure – the formal and informal culture, institutions, standards, practices, and procedures that support an enterprise – is often more important to human endeavors than hard infrastructure. For example, a highway system’s hard infrastructure (e.g., the roads) could not satisfy its enterprise – effective transportation — without the corresponding soft infrastructure of regulations, policing, driver habits, maintenance crews, and so forth. The relative cost and importance of the different aspects of infrastructure evolves, particularly during the development stages, from planning, to construction, to commissioning, operation, maintenance, and upgrading. The design, development, support, and dissemination of computer software is an archetypal example of soft infrastructure, regularly being the tail that wags the dog of computer hardware (prototypical hard infrastructure). Quantifying soft infrastructure is more difficult than hard infrastructure. Thus, it is often ignored or under-emphasized when analyzing or proposing changes to human-developed systems. In particular, government understanding and funding of scientific soft infrastructure lags well behind that of hard infrastructure (for example, experimental devices such as accelerators). In fact, the funding for soft infrastructure often only flows as a by-product of the funding for tangentially-related hard infrastructure. This paper explores the soft infrastructure of the Portable Extensible Toolkit for Scientific computing (PETSc), which we dub the PETSc community. To address the needs of next-generation science the PETSc community is simultaneously engaged in developing additional features and algorithms (construction and commissioning), supporting a vibrant set of users and developers (operation), and maintaining and upgrading existing software. The informal PETSc community agglomeration across these activities is the most valuable asset of PETSc. While we briefly discuss some of the technology applied to PETSc’s development and support, the dominant story is that of PETSc community building and strengthening, while considering the lessons we have learned that may apply to other software development projects.

The Portable Extensible Toolkit for Scientific computation (PETSc) library delivers scalable solvers for nonlinear time-dependent differential and algebraic equations and for numerical optimization.The PETSc design for performance portability addresses fundamental GPU accelerator challenges and stresses flexibility and extensibility by separating the programming model used by the application from that used by the library, and it enables application developers to use their preferred programming model, such as Kokkos, RAJA, SYCL, HIP, CUDA, or OpenCL, on upcoming exascale systems. A blueprint for using GPUs from PETSc-based codes is provided, and case studies emphasize the flexibility and high performance achieved on current GPU-based systems.

The use of multiple types of precision in mathematical software has the potential to increase its performance on new heterogeneous architectures. The xSDK project focuses both on the investigation and development of multiprecision algorithms as well as their inclusion into xSDK member libraries. This report summarizes current efforts on including and/or using mixed precision capabilities in the math libraries Ginkgo, heFFTe, hypre, MAGMA, PETSc/TAO, SLATE, SuperLU, and Trilinos, including KokkosKernels. It contains both numerical results from libraries that already provide mixed precision capabilities, as well as descriptions of the strategies to incorporate multiprecision into established libraries.