Current modeling and simulation capabilities permit tackling complex multi-physics problems, such as those encountered in ground vehicle mobility studies, using high-fidelity physics-based models for all involved subsystems, including the vehicle, tires, and deformable terrain. However, these come at significant computational burden; research and development on new software architecture and parallelization techniques is crucial in enabling such predictive simulation capabilities to be useful in design of new vehicles or in operational settings. In this paper, we describe the architecture, philosophy, and implementation of a distributed message-passing-based granular terrain simulation capability and its incorporation into an explicit force–displacement co-simulation framework to enable effective simulation of multi-physics mobility problems. We demonstrate that the proposed infrastructure has good parallel scaling characteristics and can thus effectively leverage available computing resources. Furthermore, we show that the outer communication layer, also implemented with a message passing approach, is effective and adds negligible overhead.

This study describes the implementation of a granular dynamics solver designed to run on Graphics Processing Units (GPUs). The discussion concentrates on how the Discrete Element Method (DEM) has been mapped onto the GPU architecture, the software design decisions involved in the process, and the optimizations allowed by those decisions. This solver, called Chrono::Granular, has been developed as a standalone library that can interface with other dynamics engines via triangle mesh co-simulation. A scaling analysis of the code presented herein demonstrates linear scaling with problem sizes of over two billion degrees of freedom and closing in on one billion bodies. We conclude with a study of hourglass (or hopper) mass discharge rate which compares the solver to experimental results and investigates a process for determining empirical coefficients of flow rate through simulation.

We discuss modeling, algorithmic, and software aspects that allow a simulation tool called Chrono::Granular to run billion-degree-of-freedom dynamics problems on commodity hardware, i.e., a workstation with one GPU. The ability to scale the solution to large problem sizes is traced back to an adimensionalization process combined with the use of mixed-precision data types that reduce memory pressure and improve arithmetic intensity, judicious use of the memory ecosystem on GPU cards as exposed by CUDA on Nvidia architectures, and a software implementation that prioritizes execution speed over modeling generality. The simulation approach is demonstrated for 3D scenarios with up to 710 million bodies for the frictionless case (of relevance in emulsions), and up to 210 million bodies for scenarios with friction (of relevance in terradynamics, additive manufacturing, soft-matter physics). The frictional contact model used draws on the Discrete Element Method (DEM). A performance benchmark shows linear scaling with problem size up to GPU memory capacity. The implementation has an application programming interface that enables it to interact in a cosimulation framework with third-party dynamics engines. This interaction is anchored by a force–displacement data exchange protocol that brings in external bodies as geometries defined by triangle meshes. We demonstrate the cosimulation mechanism by interfacing to an open source, multiphysics simulation engine called Chrono. Therein, triangular meshes define moving boundary conditions for Chrono::Granular, which in turn provides forces and torques acting on the triangular meshes. Several tests are considered for validation and scaling analysis purposes. The limiting aspects of the current implementation are its exclusive support of monodisperse granular systems, and its lack of handling geometries beyond spheres. These limitations are addressed by ongoing work.

In understanding the dynamics of granular systems, a discrete modeling approach tracks the motion of all particles. Such an approach is computationally demanding especially when the number of particles is large, e.g., when going beyond tens of millions of particles. In these cases, one can contemplate switching to continuummodels, which are computationally less expensive. In order to assess when such a discrete to continuum switch is justified, we report herein results for a comparison between the dynamics of granular and fluid flows obtained with an open source code that scales to handle granular systems with more than 1 billion degrees of freedom (DOFs); i.e., two orders of magnitude higher than the state of the art. On the one hand, we solve the Newton-Euler equations of motion, which govern the time evolution of the granular system. On the other hand, we solve the Navier-Stokes equations that describe the time evolution of the fluid model. Both the multibody and fluid dynamics solvers leverage parallel computing on the Graphics Processing Unit (GPU). We report similarities and differences between the dynamics of the discrete, fully resolved system and the continuum granular material model via a set of numerical experiments that include both static and highly transient scenarios. The simulation platform that anchors this contribution is publicly available on GitHub and is part of an open source code called Chrono.