The arbitrary Lagrangian-Eulerian (ALE) technique involves remapping field quantities from a Lagrangian mesh to an optimized mesh in a conservative, accurate and bounds-preserving manner. For methods based on arbitrary order finite elements, as described in [5], material volume fractions are advected in pseudo-time using flux-corrected transport (FCT) without any form of interface reconstruction. In practice, this can lead to excessive propagation of small volume fractions throughout the domain. In addition, this method requires assembly of a global advection matrix to compute the bounds-preserving low-order FCT solution. In this work, we introduce a new approach for ALE remap using a high-order matrix-free technique which incorporates a flux modification to sharpen material interfaces in a conservative manner. Our approach begins with computing a bounds-preserving low-order solution to the ALE remap equations at the element level. We then compute a sharp interface solution (not guaranteed to be bounds-preserving) which comes from solving an augmented version of the ALE remap equations with a conservative flux modification which acts to sharpen material volume fractions based on their gradients and transport directions. Using the sharp interface solution, we make global corrections to the bounds-preserving solution while maintaining preservation of bounds. By blending with the sharpened solution at the global level we are able to globally conserve mass without hindering the remap pseudo-time step. This new interface-aware ALE remap method is based entirely on partial assembly techniques where globally assembled matrix operators are no longer needed, resulting in a globally matrix-free FCT method for multi-material, multi-field ALE remap with high performance on GPU architectures. We present results of our new remap method on 1D, 2D and 3D benchmarks and describe the algorithmic tailoring for GPU architectures that was developed.

The Lawrence Livermore National Laboratory (LLNL) will soon have in place the El Capitan exascale supercomputer, based on advanced micro devices (AMD) graphics processing units (GPUs). As part of a multiyear effort under the National Nuclear Security Administration (NNSA) Advanced Simulation and Computing (ASC) program, we have been developing MARBL, a next generation, performance portable multiphysics application based on high-order finite elements. In previous years, we successfully ported the Arbitrary Lagrangian-Eulerian (ALE), multimaterial, compressible flow capabilities of MARBL to NVIDIA GPUs as described in Vargas et al. (2022, "Matrix-Free Approaches for GPU Acceleration of a High-Order Finite Element Hydrodynamics Application Using MFEM, Umpire, and RAJA," Int. J. High Perform. Comput. Appl., 36(4), pp. 492-509). In this paper, we describe our ongoing effort in extending MARBL's GPU capabilities with additional physics, including multigroup radiation diffusion and thermonuclear burn for high energy density physics (HEDP) and fusion modeling. We also describe how our portability abstraction approach based on the RAJA Portability Suite and the MFEM finite element discretization library has enabled us to achieve high performance on AMD based GPUs with minimal effort in hardware-specific porting. Throughout this work, we highlight numerical and algorithmic developments that were required to achieve GPU performance.

The formation of stars and stellar clusters remains a grand challenge problem in astrophysics that has important implications for the evolution of the interstellar medium as well as shaping the evolution of galaxies.The computational challenges are formidable and involve a coupling of highly non-linear physical processes such as hydrodynamics, self-gravity, magnetic fields, radiation transfer, supersonic turbulence, ionization, protostellar outflows, stellar winds and chemistry that have both disparate timescales as well as operate over many decades of physical length scale.These processes can regulate the feedback from nascent protostars onto the surrounding turbulent gas clouds that are the embryos of new star formation, and as a result, the feedback itself can influence the gaseous reservoir feeding newly formed protostars which in turn influence the star formation process.

The life of a multi-physics code user is complicated. Simulation crashes, efficient resource utilization, and simulation parameter choices are time consuming workflow issues that increase a user’s iteration time. Simulations often don’t provide general tools to support automatically adapting workflows to the diverse set of problems that multi-physics codes are capable of simulating. In situ visualization and analysis infrastructures are designed to be general. They are repositories of shared capability that support multiple simulation codes. Normally, the connection between simulation and in situ analysis is unidirectional (e.g., render an image or querying the mesh). In this work, we close the loop between an in situ infrastructure and a simulation, and we explore opportunities to leverage in situ triggers for automatic computational steering at runtime. We demonstrate using Ascent’s in situ trigger interface as a capable yes-no machine for controlling simulation choices.

We have conducted nineteen state-of-the-art 3D core-collapse supernova simulations spanning a broad range of progenitor masses. This is the largest collection of sophisticated 3D supernova simulations ever performed. We have found that while the majority of these models explode, not all do, and that even models in the middle of the available progenitor mass range may be less explodable. This does not mean that those models for which we did not witness explosion would not explode in Nature, but that they are less prone to explosion than others. One consequence is that the "compactness" measure is not a metric for explodability. We find that lower-mass massive star progenitors likely experience lower-energy explosions, while the higher-mass massive stars likely experience higher-energy explosions. Moreover, most 3D explosions have a dominant dipole morphology, have a pinched, wasp-waist structure, and experience simultaneous accretion and explosion. We reproduce the general range of residual neutron-star masses inferred for the galactic neutron-star population. The most massive progenitor models, however, in particular vis \`a vis explosion energy, need to be continued for longer physical times to asymptote to their final states. We find that while the majority of the inner ejecta have Y$_e = 0.5$, there is a substantial proton-rich tail. This result has important implications for the nucleosynthetic yields as a function of progenitor. Finally, we find that the non-exploding models eventually evolve into compact inner configurations that experience a quasi-periodic spiral SASI mode. We otherwise see little evidence of the SASI in the exploding models.

In diverse areas of science and technology, including inertial confinement fusion (ICF), astrophysics, geophysics, and engineering processes, turbulent mixing induced by hydrodynamic instabilities is of scientific interest as well as practical significance. Because of the fundamental roles they often play in ICF and other applications, three classes of hydrodynamic instability-induced turbulent flows-those arising from the Rayleigh-Taylor, Richtmyer-Meshkov, and Kelvin-Helmholtz instabilities-have attracted much attention. ICF implosions, supernova explosions, and other applications illustrate that these phases of instability growth do not occur in isolation, but instead are connected so that growth in one phase feeds through to initiate growth in a later phase. Essentially, a description of these flows must encompass both the temporal and spatial evolution of the flows from their inception. Hydrodynamic instability will usually start from potentially infinitesimal spatial perturbations, will eventually transition to a turbulent flow, and then will reach a final state of a true multiscale problem. Indeed, this change in the spatial scales can be vast, with hydrodynamic instability evolving from just a few microns to thousands of kilometers in geophysical or astrophysical problems. These instabilities will evolve through different stages before transitioning to turbulence, experiencing linear, weakly, and highly nonlinear states. The challenges confronted by researchers are enormous. The inherent difficulties include characterizing the initial conditions of such flows and accurately predicting the transitional flows. Of course, fully developed turbulence, a focus of many studies because of its major impact on the mixing process, is a notoriously difficult problem in its own right. In this pedagogical review, we will survey challenges and progress, and also discuss outstanding issues and future directions.

This paper describes the design and implementation of our new multigroup, multidimensional radiation hydrodynamics code F ornax and provides a suite of code tests to validate its application in a wide range of physical regimes. Instead of focusing exclusively on tests of neutrino radiation hydrodynamics relevant to the core-collapse supernova problem for which F ornax is primarily intended, we present here classical and rigorous demonstrations of code performance relevant to a broad range of multidimensional hydrodynamic and multigroup radiation hydrodynamic problems. Our code solves the comoving-frame radiation moment equations using the M 1 closure, utilizes conservative high-order reconstruction, employs semi-explicit matter and radiation transport via a high-order time stepping scheme, and is suitable for application to a wide range of astrophysical problems. To this end, we first describe the philosophy, algorithms, and methodologies of F ornax and then perform numerous stringent code tests that collectively and vigorously exercise the code, demonstrate the excellent numerical fidelity with which it captures the many physical effects of radiation hydrodynamics, and show excellent strong scaling well above 100,000 MPI tasks.

In this paper, we present the results of our three-dimensional, multi-group, multi-neutrino-species radiation/hydrodynamic simulation using the state-of-the-art code F{\sc{ornax}} of the terminal dynamics of the core of a non-rotating 16-M$_{\odot}$ stellar progenitor. The calculation incorporates redistribution by inelastic scattering, a correction for the effect of many-body interactions on the neutrino-nucleon scattering rates, approximate general relativity (including the effects of gravitational redshifts), velocity-dependent frequency advection, and an implementation of initial perturbations in the progenitor core. The model explodes within $\sim$100 milliseconds of bounce (near when the silicon-oxygen interface is accreted through the temporarily-stalled shock) and by the end of the simulation (here, $\sim$677 milliseconds after bounce) is accumulating explosion energy at a rate of $\sim$2.5$\times$10$^{50}$ ergs s$^{-1}$. The supernova explosion resembles an asymmetrical multi-plume structure, with one hemisphere predominating. The gravitational mass of the residual proto-neutron star at $\sim$677 milliseconds is $\sim$1.42 M$_{\odot}$. Even at the end of the simulation, explosion in most of the solid angle is accompanied by some accretion in an annular fraction at the wasp-like waist of the debris field. The ejecta electron fraction (Y$_e$) is distributed from $\sim$0.48 to $\sim$0.56, with most of the ejecta mass proton-rich. This may have implications for supernova nucleosynthesis, and could have a bearing on the p- and $\nu$p-processes and on the site of the first peak of the r-process. The ejecta spatial distributions of both Y$_e$ and mass density are predominantly in wide-angle plumes and large-scale structures, but are nevertheless quite patchy.

We present results of 2D axisymmetric core-collapse supernova simulations, employing the FORNAX code, of nine progenitor models spanning 12 to 25 M-circle dot. Four of the models explode with inelastic scattering off electrons and neutrons as well as the many-body correction to neutrino-nucleon scattering opacities. We show that these four models feature sharp Si-O interfaces in their density profiles, and that the corresponding dip in density reduces the accretion rate around the stalled shock and prompts explosion. The non-exploding models lack such a steep feature, highlighting the Si-O interface as one key to explosion. Furthermore, we show that all of the non-exploding models can be nudged to explosion with modest changes to macrophysical inputs, including moderate rotation and perturbations to infall velocities, as well as to microphysical inputs, including reasonable changes to neutrino-nucleon interaction rates, suggesting that all the models are perhaps close to criticality. Exploding models have energies of a few x 10(50) erg at the end of our simulation, and are rising, emphasizing the need to continue these simulations over larger grids and for longer times to reproduce the energies seen in nature. Morphology of the explosion contributes to the explosion energy, with more isotropic ejecta producing larger explosion energies. We do not find evidence for the Lepton-number Emission Self-sustained Asymmetry. Finally, we look at proto-neutron star (PNS) properties and explore the role of dimension in our simulations. We find that convection in the PNS produces larger PNS radii as well as greater 'nu(mu)' luminosities in 2D compared to 1D.

We explore with self-consistent 2D Fornax simulations the dependence of the outcome of collapse on many-body corrections to neutrino-nucleon cross sections, the nucleon-nucleon bremsstrahlung rate, electron capture on heavy nuclei, pre-collapse seed perturbations, and inelastic neutrino-electron and neutrino-nucleon scattering. Importantly, proximity to criticality amplifies the role of even small changes in the neutrino-matter couplings, and such changes can together add to produce outsized effects. When close to the critical condition the cumulative result of a few small effects (including seeds) that individually have only modest consequence can convert an anemic into a robust explosion, or even a dud into a blast. Such sensitivity is not seen in one dimension and may explain the apparent heterogeneity in the outcomes of detailed simulations performed internationally. A natural conclusion is that the different groups collectively are closer to a realistic understanding of the mechanism of core-collapse supernovae than might have seemed apparent.

For a suite of 14 core-collapse models during the dynamical first second after bounce, we calculate the detailed neutrino 'light' curves expected in the underground neutrino observatories Super-Kamiokande, DUNE, JUNO, and IceCube. These results are given as a function of neutrino-oscillation modality (normal or inverted hierarchy) and progenitor mass (specifically, post-bounce accretion history), and illuminate the differences between the light curves for ID (spherical) models that don't explode with the corresponding 2D (axisymmetric) models that do. We are able to identify clear signatures of explosion (or non-explosion), the post-bounce accretion phase, and the accretion of the silicon/oxygen interface. In addition, we are able to estimate the supernova detection ranges for various physical diagnostics and the distances out to which various temporal features embedded in the light curves might be discerned. We find that the progenitor mass density profile and supernova dynamics during the dynamical explosion stage should be identifiable for a supernova throughout most of the Galaxy in all the facilities studied and that detection by any one of them, but in particular more than one in concert, will speak volumes about the internal dynamics of supernovae.

Momentum deposition by radiation pressure from young, massive stars may help to destroy molecular clouds and unbind stellar clusters by driving large-scale outflows. We extend our previous numerical radiation hydrodynamic study of turbulent star-forming clouds to analyze the detailed interaction between non-ionizing UV radiation and the cloud material. Our simulations trace the evolution of gas and star particles through self-gravitating collapse, star formation, and cloud destruction via radiation-driven outflows. These models are idealized in that we include only radiation feedback and adopt an isothermal equation of state. Turbulence creates a structure of dense filaments and large holes through which radiation escapes, such that only similar to 50% of the radiation is (cumulatively) absorbed by the end of star formation. The surface density distribution of gas by mass as seen by the central cluster is roughly lognormal with sigma(ln) (Sigma) = 1.3-1.7, similar to the externally projected surface density distribution. This allows low surface density regions to be driven outwards to nearly 10 times their initial escape speed nu(esc). Although the velocity distribution of outflows is broadened by the lognormal surface density distribution, the overall efficiency of momentum injection to the gas cloud is reduced because much of the radiation escapes. The mean outflow velocity is approximately twice the escape speed from the initial cloud radius. Our results are also informative for understanding galactic-scale wind driving by radiation, in particular, the relationship between velocity and surface density for individual outflow structures and the resulting velocity and mass distributions arising from turbulent sources.

We present new 1D (spherical) and 2D (axisymmetric) simulations of electron-capture (EC) and low-mass iron-core-collapse supernovae (SN). We consider six progenitor models: the ECSN progenitor from Nomoto; two ECSN-like low-mass low-metallicity iron-core progenitors from A. Heger (2016, private communication); and the 9, 10, and 11 M-circle dot (zero-age main-sequence) progenitors from Sukhbold et al. We confirm that the ECSN and ESCN-like progenitors explode easily even in 1D with explosion energies of up to a 0.15 Bethes (1 B equivalent to 10(51) erg), and are a viable mechanism for the production of very-low-mass neutron stars. However, the 9, 10, and 11 M-circle dot progenitors do not explode in 1D and are not even necessarily easier to explode than higher-mass progenitor stars in 2D. We study the effect of perturbations and of changes to the microphysics and we find that relatively small changes can result in qualitatively different outcomes, even in 1D, for models sufficiently close to the explosion threshold. Finally, we revisit the impact of convection below the protoneutron star (PNS) surface. We analyze 1D and 2D evolutions of PNSs subject to the same boundary conditions. We find that the impact of PNS convection has been underestimated in previous studies and could result in an increase of the neutrino luminosity by up to factors of two.

We present an implementation of an adaptive ray-tracing (ART) module in the Athena hydrodynamics code that accurately and efficiently handles the radiative transfer involving multiple point sources on a three-dimensional Cartesian grid. We adopt a recently proposed parallel algorithm that uses nonblocking, asynchronous MPI communications to accelerate transport of rays across the computational domain. We validate our implementation through several standard test problems, including the propagation of radiation in vacuum and the expansions of various types of H ii regions. Additionally, scaling tests show that the cost of a full ray trace per source remains comparable to that of the hydrodynamics update on up to ∼ 10 3 processors. To demonstrate application of our ART implementation, we perform a simulation of star cluster formation in a marginally bound, turbulent cloud, finding that its star formation efficiency is 12% when both radiation pressure forces and photoionization by UV radiation are treated. We directly compare the radiation forces computed from the ART scheme with those from the M 1 closure relation. Although the ART and M 1 schemes yield similar results on large scales, the latter is unable to resolve the radiation field accurately near individual point sources.

We perform the first self-consistent, time-dependent, multi-group calculations in two dimensions (2D) to address the consequences of using the ray-by-ray+ transport simplification in core-collapse supernova simulations. Such a dimensional reduction is employed by many researchers to facilitate their resource-intensive calculations. Our new code (Fornax) implements multi-D transport, and can, by zeroing out transverse flux terms, emulate the ray-by-ray+ scheme. Using the same microphysics, initial models, resolution, and code, we compare the results of simulating 12-, 15-, 20-, and 25-M_⊙ progenitor models using these two transport methods. Our findings call into question the wisdom of the pervasive use of the ray-by-ray+ approach. Employing it leads to maximum post-bounce/pre-explosion shock radii that are almost universally larger by tens of kilometers than those derived using the more accurate scheme, typically leaving the post-bounce matter less bound and artificially more "explodable." In fact, for our 25-M_⊙ progenitor, the ray-by-ray+ model explodes, while the corresponding multi-D transport model does not. Therefore, in two dimensions the combination of ray-by-ray+ with the axial sloshing hydrodynamics that is a feature of 2D supernova dynamics can result in quantitatively, and perhaps qualitatively, incorrect results.

Radiation feedback from stellar clusters is expected to play a key role in setting the rate and efficiency of star formation in giant molecular clouds. To investigate how radiation forces influence realistic turbulent systems, we have conducted a series of numerical simulations employing the Hyperion radiation hydrodynamics solver, considering the regime that is optically thick to ultraviolet and optically thin to infrared radiation. Our model clouds cover initial surface densities between Sigma(cl),0 similar to 10-300 M-circle dot pc(-2), with varying initial turbulence. We follow them through turbulent, self-gravitating collapse, star cluster formation, and cloud dispersal by stellar radiation. All our models display a log-normal distribution of gas surface density Sigma; for an initial virial parameter alpha(vir,0) = 2, the log-normal standard deviation is sigma(ln) (Sigma) = 1-1.5 and the star formation rate coefficient epsilon(ff),((rho) over bar) = 0.3-0.5, both of which are sensitive to turbulence but not radiation feedback. The net star formation efficiency (SFE) epsilon(final) increases with Sigma(cl),(0) and decreases with alpha(vir,0). We interpret these results via a simple conceptual framework, whereby steady star formation increases the radiation force, such that local gas patches at successively higher Sigma become unbound. Based on this formalism (with fixed sigma(ln) (Sigma)), we provide an analytic upper bound on epsilon(final), which is in good agreement with our numerical results. The final SFE depends on the distribution of Eddington ratios in the cloud and is strongly increased by the turbulent compression of gas.

Radiation feedback from young star clusters embedded in giant molecular clouds (GMCs) is believed to be important to the control of star formation. For the most massive and dense clouds, including those in which super star clusters (SSCs) are born, pressure from reprocessed radiation exerted on dust grains may disperse a significant portion of the cloud mass back into the interstellar medium. Using our radiation hydrodynamics code, Hyperion, we conduct a series of numerical simulations to test this idea. Our models follow the evolution of self-gravitating, strongly turbulent clouds in which collapsing regions are replaced by radiating sink particles representing stellar clusters. We evaluate the dependence of the star formation efficiency (SFE) on the size and mass of the cloud and., the opacity of the gas to infrared (IR) radiation. We find that the single most important parameter determining the evolutionary outcome is kappa, with kappa greater than or similar to 15 cm(2)g(-1) needed to disrupt clouds. For kappa = 20-40 cm(2)g(-1), the resulting SFE= 50%-70% is similar to empirical estimates for some SSC-forming clouds. The opacities required for GMC disruption likely apply only in dust-enriched environments. We find that the subgrid model approach of boosting the direct radiation force L c by a "trapping factor" equal to a cloud's mean IR optical depth can overestimate the true radiation force by factors of similar to 4-5. We conclude that feedback from reprocessed IR radiation alone is unlikely to significantlyreduce star formation within GMCs unless their dust abundances or cluster light-to-mass ratios are enhanced.