Divide-Conquer-Recombine: An Algorithmic Pathway Toward Metascalability

We propose an extension of the divide-and-conquer (DC) algorithmic paradigm called divide-conquer-recombine (DCR) in order to develop O(N) applications that will continue to scale on future parallel supercomputing, i.e., making them metascalable (or "design once, scale on new architectures"). In DCR, the DC phase constructs globally informed local solutions, which in the recombine phase are synthesized into a global solution. Innovative recombination algorithms allow the synthesis of a variety of global properties in broad applications. To enable large spatial-scale molecular dynamics (MD) simulations, DCR-in-space is empowered by globally scalable and locally fast (GSLF) hybrid solvers based on spatial locality. In addition, DCR-in-time is used to predict long-time dynamics based on temporal locality, while utilizing space-time-ensemble parallelism (STEP). We have used DCR to perform quantum molecular dynamics (QMD) and reactive molecular dynamics (RMD) simulations that encompass unprecedented spatiotemporal scales. Our 50.3 million-atom QMD benchmark achieved a parallel efficiency of 0.984 and 50.5% of the peak floating-point performance on 786,432 IBM Blue Gene/Q cores. Production QMD simulation involving 16,661 atoms for 21,140 time steps (or 129,208 self-consistent-field iterations) revealed a novel nanostructural design for on-demand hydrogen production from water, advancing renewable energy technologies. Nonadiabatic QMD simulation of photoexcitation dynamics involving 6,400 atoms reached the experimental time scales, elucidating molecular mechanisms of a novel singlet-fission phenomenon to realize low-cost, high-efficiency solar cells. Our billion-atom RMD simulation revealed the role of focused nanojet for the damage of solid surface caused by shock-induced collapse of nanobubbles in water, and suggested how to mitigate the damage by filling the bubble with inert gas.