The steadily increasing power of supercomputing systems is enabling very high resolution simulations of compressible, turbulent flows in the high Reynolds number limit, which is of interest in astrophysics as well as in several other fluid dynamical applications. This paper discusses two such simulations, using grids of up to 8 billion cells. In each type of flow, convergence in a statistical sense is observed as the mesh is refined. The behavior of the convergent sequences indicates how a subgrid-scale model of turbulence could improve the treatment of these flows by high-resolution Euler schemes like PPM. The best resolved case, a simulation of a Richtmyer-Meshkov mixing layer in a shock tube experiment, also points the way toward such a subgrid-scale model. Analysis of the results of that simulation indicates a proportionality relationship between the energy transfer rate from large to small motions and the determinant of the deviatoric symmetric strain as well as the divergence of the velocity for the large-scale field.

Understanding turbulence and mix in compressible flows is of fundamental importance to real-world applications such as chemical combustion and supernova evolution. The ability to run in three dimensions and at very high resolution is required for the simulation to accurately represent the interaction of the various length scales, and consequently, the reactivity of the intermixin species. Toward this end, we have carried out a very high resolution (over 8 billion zones) 3-D simulation of the Richtmyer-Meshkov instability and turbulent mixing on the IBM Sustained Stewardship TeraOp (SST) system, developed under the auspices of the Department of Energy (DOE) Accelerated Strategic Computing Initiative (ASCI) and located at Lawrence Livermore National Laboratory. We have also undertaken an even higher resolution proof-of-principle calculation (over 24 billion zones) on 5832 processors of the IBM system, which executed for over an hour at a sustained rate of 1.05 Tflop/s, as well as a short calculation with a modified algorithm that achieved a sustained rate of 1.18Tflop/s. The full production scientific simulation, using a further modified algorithm, ran for 27,000 timesteps in slightly over a week of wall time using 3840 processors of the IBM system, clockin a sustained throughput of roughly 0.6 teraflop per second (32-bit arithmetic). Nearly 300,000 graphics files comprising over three terabytes of data were produced and post-processed. The capability of running in 3-D at high resolution enabled us to get a more accurate and detailed picture of the fluid-flow structure - in particular, to simulate the development of fine scale structures from the interactions of long-and short-wavelength phenomena, to elucidate differences between two-dimensional and three-dimensional turbulence, to explore a conjecture regarding the transition from unstable flow to fully developed turbulence with increasing Reynolds number, and to ascertain convergence of the computed solution with respect to mesh resolution.

In 1993, members of our team collaborated with Silicon Graphics to perform the first full-scale demonstration of the computational power of the SMP cluster supercomputer architecture. That demonstration involved the simulation of homogeneous, compressible turbulence on a uniform grid of a billion cells, using our PPM gas dynamics code. This computation was embarrassingly parallel, the ideal test case, and it achieved only 4.9 Gflop/s performance, slightly over half that achievable by this application on the most expensive supercomputers of that day. After four to five solid days of computation, when the prototype machine had to be dismantled, the simulation was only about 20% completed. Nevertheless, this computation gave us important new insights into compressible turbulence and also into a powerful new mode of cost-effective, commercially sustainable supercomputing [S]. In the intervening 6 years, the SMP cluster architecture has become a fundamental strategy for several large supercomputer centers in the US, including the DOE's ASCI centers at Los Alamos National Laboratory and at the Lawrence Livermore National Laboratory and the NSF's center NCSA at the University of Illinois. This SMP cluster architecture now underlies product offerings at the high-end of performance from SGI, IBM, and HP, among others. Nevertheless, despite many successes, it is our opinion that the computational science community is only now beginning to exploit the full promise of these new computing platforms. In this paper, we will briefly discuss two key architectural issues, vector computing and the flat multiprocessor architecture, which continue to drive spirited discussions among computational scientists, and then we will describe the hierarchical shared memory programming paradigm that we feel is best suited to the creative use of SMP cluster systems. Finally, we will give examples of recent large-scale simulations carried out by our team on these kinds of systems and point toward the still more challenging work which we foresee in the near future.

The new generation of powerful DSM and SMP cluster computers enables simulations of fluid dynamics at sufficient resolution to compute the complex nonlinear interactions of small-scale turbulent motions with a large-scale driving flow. With a new programming model of hierarchical shared memory multitasking, it is possible to exploit these new systems without disrupting the flow of small and medium-sized jobs that makes their existence possible.

Three-dimensional high-resolution simulations are performed of the Richtmyer-Meshkov (RM) instability for a Mach 6 shock, and of the passage of a second shock from the same side through a developed RM instability. The second shock is found to rapidly smear fine structure and strongly enhance mixing. Studies of the interaction of moderately strong shocks with a pre-existing turbulent field indicate amplification of transverse vorticity and reduction of stream-wise vorticity, as well as the...

Preliminary results of three-dimensional simulations of compressible Rayleigh-Taylor instability and turbulent mixing in an ideal gas using the piecewise-parabolic method (PPM) (with and without molecular dissipation terms) are presented. Simulations with spatial resolutions up to 512{sup 3} were performed. Two types of convergence studies are presented. Statistical analyses of the data are discussed, include: 1: spectra, and; 2) horizontally-averaged terms in the kinetic energy and onstrophy density evolution equations. The application of this statistical data to the development and testing of subgrid-scale models appropriate for compressible Rayleigh-Taylor instability-induced turbulent mixing is discussed.