The accurate representation of a problem domain in terms of a geometric model and its effective discretization into a mesh plays an important role in achieving higher-quality simulation results and better computational efficiency. In this work, recent developments to extend an automated tokamak modeling and meshing infrastructure to better support the near flux field following meshing requirements of the XGC Gyro-kinetic Code are presented. The first extension is a procedure to effectively represent a general set of O-point and X-points configurations. The second extension is a procedure to decompose the geometric model that includes the tokamak wall, selected flux curves and the separatrix curves in a manner such that an appropriate set of mesh generation procedures can be applied to each subregion. The third extension is the application of appropriate mesh generation procedures in each subregion to create the desired final mesh.
This article presents M u MF i M, an open-source application for multiscale modeling of fibrous materials on massively parallel computers. M u MF i M uses two scales to represent fibrous materials such as biological network materials (extracellular matrix, connective tissue, etc.). It is designed to make use of multiple levels of parallelism, including distributed parallelism of the macro- and micro-scales as well as GPU-accelerated data-parallelism of the microscale. Scaling results of the GPU accelerated microscale show that solving microscale problems concurrently on the GPU can lead to a 1000x speedup over the solution of a single RVE on the GPU. In addition, we show nearly optimal strong and weak scaling results of M u MF i M on up to 128 nodes of AiMOS (Rensselaer Polytechnic Institute) which is composed of IBM AC922 nodes with 6 Volta V100 GPU and 2 20 core Power 9 CPUs each. We also show how M u MF i M can be used to solve problems of interest to the broader engineering community, in particular providing an example of the facet capsule ligament (FCL) of the human spine undergoing uniaxial extension.
EDITORIAL article Front. Psychol., 20 July 2023Sec. Cognition Volume 14 - 2023 | https://doi.org/10.3389/fpsyg.2023.1236748
The growing scale of high-performance computing systems increasingly enables scientists to develop more complex applications as in situ workflows composed of coupled simulation and analysis codes. It is therefore important that workflow programming systems and runtime middleware support the composition and execution of these complex applications intuitively and efficiently. The scientific computing community has put significant effort into purpose-built coupled simulation codes that have been optimized for specialized use cases. However, the development effort involving the coupling of established codes has been largely ad hoc. The Benesh programming system was recently proposed to support the development of coupled simulation workflows from existing code bases. Benesh allows a shared data model to be defined across established codes, so that they can be interfaced in a flexible, coupled workflow. In this paper, we develop Benesh into a workflow development framework. Using Benesh, we develop workflow data model and data exchange definitions for coupled, in situ workflows. We evaluate the cost of development using Benesh in terms of development time and overhead, showing that Benesh offers development advantages without undue impact upon workflow performance.
This paper presents XGCm, a new unstructured mesh gyrokinetic Particle-in-Cell (PIC) code for modeling fusion plasma. The physical models and aspects of the numerical methods employed are the same as those used in the X-point gyrokinetic code, XGC. The core difference is that XGCm builds on an unstructured mesh-centric infrastructure that supports a distributed mesh, making it scalable in both the number of mesh elements and number of particles. A second advantage of an unstructured mesh infrastructure is its performance is not degraded when generally graded or anisotropic meshes are used. The switch from a particle-centric data infrastructure to a distributed mesh-centric infrastructure required the introduction of new methods to execute core PIC operations and substantial modifications to a number of key algorithms from those used in XGC. First, we present the methods and algorithms used in the development of XGCm, which performs all key computing steps on the GPU accelerators. GPU accelerators are providing the main computational power of current generation U.S. Department of Energy supercomputers. Secondly, we perform code validation and test using the circular geometry cyclone base case and realistic DIII-D geometry case, respectively. The turbulence growth rate shows excellent agreement with existing XGC result in the first case, while ion temperature gradient turbulence growth is further demonstrated in the second case. Finally, we present weak scaling results, using up to full system (27,648 GPUs) of the Oak Ridge National Laboratory's Summit supercomputer.
A fully 3D unstructured mesh based global impurity transport code, GITRm is presented in this paper. It is a high-performance Monte Carlo particle (neutral atom and ion) tracking code, based on the trace approximation, to simulate the erosion, ionization, migration, and redistribution of material from plasma-facing components in magnetically confined fusion devices. It is designed to target complex geometries including non-axisymmetric local features such as bumpers, probes, tile gaps etc., and uses strongly graded and anisotropic elements to accurately represent the plasma fields. GITRm is built on the PUMIPic infrastructure [1], executes using distributed meshes and is performant on GPU accelerated computer systems. Three example cases, including a weak scaling study with about 1.5 billion particles on up to 144 GPUs, are used to demonstrate the utility of GITRm.
Many engineering problems are characterized as having complex geometry that evolves over time combined with complex physical behaviors. In addition to an appropriate combination of physics models, discretization techniques and numerical methods, the effective simulation of such problems requires a set of methods to track the evolution of the simulation domain and the spatial decomposition of that domain into the desired meshes. This paper presents a set of geometry model representation and mesh generation/update procedures developed, and combined, to support the simulation of evolving geometry problems. The geometry representation and tracking procedures presented are designed to effectively interact with complex CAD representations while providing an effective means to account for sliding interfaces for which the base model topology changes. The mesh generation procedures combine automatic boundary layer and unstructured mesh generation with procedures to effectively generate semi-structured meshes in confined regions of geometry that can include step variations. The mesh update procedures combine mesh motion and local cavity mesh modification to provide well controlled, and adapted, meshes throughout an evolving geometry simulation.
Predictive modeling of the pitch-walk variance from multistep coupled processes, such as SAQP using experimental metrology observables, has the potential to give both deep understanding and a control mechanism for pitch-walk variance. In this study, with the Bayesian dropout approximation, a methodology using Bayesian inference via use of stochastic neural networks was employed to both model and predict the SAQP pitch-walk variance distribution. Bayesian neural networks were implemented as variational ensembles of networks with hidden layers, where the neural net training uses conventional dropout, while the forward solves employ a dropout Bayesian vector methodology previously developed by Gal and Ghahramani.1, 2 An important distinction here is that the forward propagations effectively sample the network to make a prediction, resulting in a distribution of outputs achieving the best model, not just a single expectation value. A complete dataset of fin module OCD metrology measurements per chip at top mandrel, bottom mandrel, and final fin reveal were used. Since the measured dataset was limited to small number chip locations, data augmentation with the highly efficient method of the volume of simplex was used to generate 30K samples. The synthetic data and the experimental data were used for neural network calibration and validation, respectively.
Unstructured mesh particle-in-cell, PIC, simulations executing on the current and next generation of massively parallel systems require new methods for both the mesh and particles to achieve performance and scalability on GPUs. The traditional approach to implementing PIC simulations defines data structures and algorithms in terms of particles with a full copy of the unstructured mesh on every process. To effectively scale the unstructured mesh and particles, mesh-based PIC uses the unstructured mesh as the predominant data structure with the particles stored in terms of the mesh entities. This paper details the PUMIPic library, a framework for developing efficient and performance-portable mesh-based PIC simulations on GPU systems. A pseudo physics simulation based on a five-dimensional gyro-kinetic code for modeling plasma physics is used to examine the performance of PUMIPic. Scaling studies of the unstructured mesh partition and number of particles are performed up to 4096 nodes of the Summit system at Oak Ridge National Laboratory. The studies show that mesh-based PIC can utilize a partitioned mesh and maintain scaling up to system limitations.
This paper presents a parallel interface tracking approach for evolving geometry problems where both the computational domain and mesh are updated as dictated by the analysis. An interface-fitted conforming hybrid/mixed mesh with anisotropic layered elements is used. A combination of mesh motion and mesh modification is employed to update the mesh to account for the interface motion. Mesh modification is triggered only when necessary. During mesh motion and modification the desired structure, shape and resolution of the anisotropic layered elements at the interface are maintained. All steps are performed on partitioned meshes on distributed-memory parallel computers. The effectiveness of the current approach is demonstrated on two problems with large motion or deformation in the geometry.
The goal of this milestone was to improve the high-order software ecosystem for CEED-enabled ECP applications by making progress on efficient matrix-free kernels targeting forthcoming ECP architectures. These kernels included matrix-free preconditioning and the development of new set of CEED solver bake-off problems. As part of this milestone, we also released the next version of the CEED software stack, CEED-4.0, reported on results from several application collaborations, and documented the efforts of porting to AMD GPUs for Frontier and other modern architectures, such as Fugaku. The specific tasks addressed in this milestone were: (1) Port and run CEED benchmarks/miniapps on Frontier EA systems; (2) Demonstrate performant libCEED integration in MFEM, Nek and applications; (3) Matrix-free preconditioning of high-order operators; (4) Benchmark problems for fast high-order solvers on GPU platforms; and (5) Public release of CEED-4.0. The artifacts delivered include the next version of the CEED software stack, CEED-4.0, the next libCEED release, libCEED-0.8, and a number of developments integrated within applications to improve their GPU and CPU performance and capabilities. See the CEED website, https://ceed.exascaleproject.org and the CEED GitHub organization, https://github.com/ceed for more details.
We know about what the public thinks of PMQs, and we know an increasing amount about questions to PMs. However, very little is known about the nature and state of questions that the PM asks during the procedure. Controlling for the average length of PMQs, we find that while the average number of Order Paper questions reached has stayed relatively static, the total number of questions asked has nearly halved over time while questions asked by the PM (including questions levelled at both the Leader of the Opposition and the opposition) have increased over time. We also find that specific blaming by the PM of the previous government is far higher right after party rotation in government (Blair and Cameron) than after PM rotation intra-party government (Brown and May). Finally, we find that Cameron was significantly more likely to ask questions during PMQs when this correlated with higher public opinion net approval ratings.
The simulation of phase change processes that occur at high rates, like the collapse of a vapor bubble or the combustion of dense energetic materials, poses significant challenges that include strong discontinuities in select field variables at the interface, high-speed flows in at least one phase, significant role of compressibility, disparate phase rate and acoustic time scales and advection dominated processes. In this paper we present a finite element based method that addresses these challenges. The discretization is continuous everywhere except at the interface and it inherits its stability properties from both continuous and discontinuous finite element formulations like the SUPG and interior penalty methods. We track the evolution of the interface mesh and accommodate its motion in the volume by moving the mesh in accordance with an elastic analogy within an arbitrary eulerian lagrangian (ALE) framework. This motion is interspersed with a few select steps of mesh modification. We demonstrate that the proposed method has desirable discrete conservation properties and outline a proof for its stability. We also describe how this method is implemented in a finite element code within an implicit predictor-corrector time stepping scheme. Finally we apply this method to a series of phase change problems involving an energetic material, where we verify its implementation and demonstrate its utility.
Prior to the creation of the committee system and the Scottish Parliament, a Consultative Steering Group was tasked with drawing up detailed proposals of how the Scottish Parliament was going to operate, and from the outset, committees were expected to be the ‘engine rooms’ not only for the effective scrutiny and influence of government, but for policy initiation as well. The idea was for a ‘new politics’ in which the ‘old politics’ of Westminster (majoritarian and top-down) would be replaced by more consensus-based and bottom-up democracy. To that end, committees were given extensive powers to scrutinise policy, administration and finance, to conduct inquiries, and to introduce legislation.