Quantum computers must operate in concert with classical computers to deliver on the promise of quantum advantage for practical problems. To achieve that, it is important to understand how quantum and classical computing can interact together, and how one can characterize the scalability and efficiency of hybrid quantum–classical workflows. So far, early experiments with quantum-centric supercomputing workflows have been limited in scale and complexity. Here, we use a Heron quantum processor deployed on premises with the entire supercomputer Fugaku to perform the largest computation of electronic structure involving quantum and classical high-performance computing. We design a closed-loop workflow between the quantum processors and 152,064 classical nodes of Fugaku, to approximate the electronic structure of chemistry models beyond the reach of exact diagonalization, with accuracy comparable to some all-classical approximation methods. Our work pushes the limits of the integration of quantum and classical high-performance computing, showcasing computational resource orchestration at the largest scale possible for current classical supercomputers.

Molten salts such as FLiBe (2LiF–BeF_2) are leading blanket materials for breeding and recovering tritium in fusion reactors. Predicting tritium speciation requires accurate electronic ground-state energies for representative molten-salt clusters, a demanding task for correlated electronic-structure methods. Here we report the first application of heterogeneous quantum–classical computing to tritium binding in FLiBe. Clusters drawn from ab initio molecular dynamics are partitioned by an embedded-wavefunction (EWF) method into atom-centered fragments, and the largest fragments are solved on IBM quantum hardware using extended sample-based quantum diagonalization (ext-SQD). Across nine clusters, the heterogeneous quantum–classical workflow reproduces fragment ground-state energies with agreement to full configuration interaction within 0.7 kcal/mol and a mean absolute deviation of 0.3 kcal/mol. In contrast, fragmented and unfragmented conformational energy differences and tritium binding energies differ by 12 kcal/mol and 110 kcal/mol on average, respectively, identifying fragment construction rather than fragment solution as the dominant source of algorithmic bias. To the best of our knowledge, this is the first such demonstration for a charged ionic system and in particular an inorganic molten salt, where electrostatic and polarization effects make the accurate treatment of electronic correlation particularly challenging. These results also identify areas of future research towards an accurate and scalable quantum–classical workflow to compute free-energy estimates of tritium speciation in fusion blankets.

We develop a workflow decomposing a molecule into fragments via quantum embedding and simulating them with a heterogeneous quantum-classical (HQC) method. We sample fragment electronic configurations on two 156-qubit quantum processors (ibm_cleveland, ibm_kobe), using up to 94 qubits, running 21,006 circuits for over 239 hours, collecting 3.0 · 10^9 measurement outcomes – the most resource-intensive HQC computation for quantum chemistry to date. We compute fragment wavefunctions via optimized subspace diagonalization on supercomputers Fugaku, Miyabi-G, and ROQUO, achieving 72.5% parallel efficiency with scalable distributed linear algebra kernels. We simulate two protein-ligand complexes spanning dispersion- and electrostatics-dominated regimes (11,608, 12,635 atoms), demonstrate >40× increase in system size and up to 210× improvement in accuracy over previous state-of-the-art, with HQC matching coupled-cluster (CCSD) accuracy in fragment energies. We present the first HQC protein-ligand binding prediction using a mixed-basis set and an automated end-to-end workflow enabling practical HQC calculations of large protein systems.

Hybrid quantum-HPC algorithms advance research by delegating complex tasks to quantum processors and using HPC systems to orchestrate workflows and complementary computations. Sample-based quantum diagonalization (SQD) is a hybrid quantum-HPC method in which information from a molecular Hamiltonian is encoded into a quantum circuit for evaluation on a quantum computer. A set of measurements on the quantum computer yields electronic configurations that are filtered on the classical computer, which also performs diagonalization on the selected subspace and identifies configurations to be carried over to the next step in an iterative process. Diagonalization is the most demanding task for the classical computer. Previous studies used the Fugaku supercomputer and a highly scalable diagonalization code designed for CPUs. In this work, we describe our efforts to enable efficient scalable and portable diagonalization on heterogeneous systems using GPUs as the main compute engines based on the previous work. GPUs provide massive on-device thread-level parallelism that is well aligned with the algorithms used for diagonalization. We focus on the computation of ground-state energies and wavefunctions using the Davidson algorithm with a selected set of electron configurations. We describe the offload strategy, code transformations, and data-movement, with examples of measurements on the Frontier supercomputer and five other GPU accelerated systems. Our measurements show that GPUs provide an outstanding performance boost of order 100x on a per-node basis. This dramatically expedites the diagonalization step-essential for extracting ground and excited state energies-bringing the classical processing time down from hours to minutes.

Vela is a cloud-native system designed for LLM training workloads built using off-the-shelf hardware, Linux KVM-based virtualization, and a virtualized RDMA over Converged Ethernet (RoCE) network. Vela virtual machines (VMs) support peer-to-peer DMA between the GPUs and SRIOV-based network interface. In this paper, we share Vela's key architectural aspects with details from an NVIDIA A100 GPU-based deployment in one of the IBM Cloud data centers. Throughout the paper, we share insights and experiences from designing, building, and operating the system over a similar to 2.5 year timeframe to highlight the capabilities of readily available software and hardware technologies and the improvement opportunities for future AI systems, thereby making AI infrastructure more accessible to a broader community. As we evaluated the system for performance at similar to 1500 GPU scale, we achieved similar to 80% of the ideal throughput while training a 50 billion parameter decoder model using model parallelism, and similar to 70% per GPU FLOPS compared to a single VM with the High-Performance Linpack benchmark.

AI Infrastructure plays a key role in the speed and cost-competitiveness of developing and deploying advanced AI models. The current demand for powerful AI infrastructure for model training is driven by the emergence of generative AI and foundational models, where on occasion thousands of GPUs must cooperate on a single training job for the model to be trained in a reasonable time. Delivering efficient and high-performing AI training requires an end-to-end solution that combines hardware, software and holistic telemetry to cater for multiple types of AI workloads. In this report, we describe IBM's hybrid cloud infrastructure that powers our generative AI model development. This infrastructure includes (1) Vela: an AI-optimized supercomputing capability directly integrated into the IBM Cloud, delivering scalable, dynamic, multi-tenant and geographically distributed infrastructure for large-scale model training and other AI workflow steps and (2) Blue Vela: a large-scale, purpose-built, on-premises hosting environment that is optimized to support our largest and most ambitious AI model training tasks. Vela provides IBM with the dual benefit of high performance for internal use along with the flexibility to adapt to an evolving commercial landscape. Blue Vela provides us with the benefits of rapid development of our largest and most ambitious models, as well as future-proofing against the evolving model landscape in the industry. Taken together, they provide IBM with the ability to rapidly innovate in the development of both AI models and commercial offerings.

HPC (high performance computing) applications come with a variety of requirements for computation, communication, and storage; and many of these requirements can be met with commodity technology available in public clouds. In this article, we report on results for several well-known HPC applications on IBM public cloud, and we describe best practices for running such applications on cloud systems in general. Our results show that public clouds are not only ready for HPC workloads, but they can provide performance comparable to, and in some cases better than, current supercomputers.

In this paper, we explain the techniques used in IBM Blue Gene®/Q Compute chips to achieve high energy efficiency. Architectural techniques include the choice of a power-efficient, throughput-oriented processor core with a SIMD (single-instruction, multiple-data) floating-point unit, as well as multiple frequency domains for moving data. Design techniques include clock gating and the use of multiple threshold voltage devices. From a systems perspective, power is reduced by using a speed binning technique that characterizes the manufacturing variability of chips during wafer test, permitting similar chips to be packaged on the same board and run at the lowest voltage possible. We describe the techniques used to monitor and manage the power and performance of the various subunits of the Blue Gene/Q chip. Details include the functioning of the environmental monitor and the performance counters. Using these facilities, we describe the framework to understand how the chip’s subunits contribute to the total active and leakage power consumed. A power characterization technique for the development of application-dependent power projection models is presented. Differences between estimated power before chip tape-out versus measured power are discussed.

This paper provides a short overview of the IBM Blue Gene® project and an introduction to all of the papers in this issue of the IBM Journal of Research and Development.

This paper explores the performance and optimization of the IBM Blue Gene/Q (BG/Q) five dimensional torus network on up to 16K nodes. The BG/Q hardware supports multiple dynamic routing algorithms and different traffic patterns may require different algorithms to achieve best performance. Between 85% to 95% of peak network performance is achieved for all-to-all traffic, while over 85% of peak is obtained for challenging bisection pairings. A new software-controlled algorithm is developed for bisection traffic that selects which hardware algorithm to employ and achieves better performance than any individual hardware algorithm. The benefit of dynamic routing is shown for a highly non-uniform "transpose" traffic pattern. To evaluate memory and network performance, the HPCC Random Access benchmark was tuned for BG/Q and achieved 858 Giga Updates per Second (GUPS) on 16K nodes. To further accelerate message processing, the message libraries on BG/Q enable the offloading of messaging overhead onto dedicated communication threads. Several applications, including Algebraic Multigrid (AMG), exhibit from 3 to 20% gain using communication threads.

This paper examines the communication characteristics of a collection of scientific applications selected from the LLNL's Sequoia suite of benchmarks and the ANL's workload. By using an instrumentation library built on top of MPI we collect and characterize the applications's messaging behavior: the type of communication patterns and primitives used, the amount of time spent for communication, the message sizes, the total amount of data exchanged, and the impact of collective primitives, through communication matrices we visualize the actual communication patterns to highlight symmetries and other relevant peculiarities. Our analysis exposes several similarities between the applications -- namely the utilization of common low-dimensional stencils, and the use of a small set of collective primitives, in particular all-reduces with small vectors. Overall, our study provides a better understanding of the communication characteristics of several important scientific applications and benchmarks.

The Weather Research and Forecast (WRF) model is a limited-area model of the atmosphere for mesoscale research and operational numerical weather prediction (NWP). A petascale problem is a WRF nature run that provides very high-resolution "truth" against which more coarse simulations or perturbation runs may be compared for purposes of studying predictability, stochastic parameterization, and fundamental dynamics. We carried out a nature run involving an idealized high resolution rotating fluid on the hemisphere to investigate scales that span the k-3 to k-5/3 kinetic energy spectral transition of the observed atmosphere using 65,536 processors of the BG/L machine at LLNL. We worked through issues of parallel I/O and scalability. The primary result is not just the scalability and high Tflops number, but an important step towards understanding weather predictability at high resolution.

We present a novel performance monitor architecture, implemented in the Blue Gene/PTM supercomputer. This performance monitor supports the tracking of a large number of concurrent events by using a hybrid counter architecture. The counters have their low order data implemented in registers which are concurrently updated, while the high order counter data is maintained in a dense SRAM array that is updated from the registers on a regular basis. The per formance monitoring architecture includes support for per- event thresholding and fast event notification, using a two- phase interrupt-arming and triggering protocol. A first implementation provides 256 concurrent 64b counters which offers an up to 64x increase in counter number compared to performance monitors typically found in microprocessors today, and thereby dramatically expands the capabilities of counter-based performance tuning.

As the global energy economy makes the transition from fossil fuels toward cleaner alternatives, nuclear fusion becomes an attractive potential solution for satisfying growing needs. Fusion, the power source of the stars, has been the focus of active research since the early, 1950s. While progress has been impressive--especially for magnetically confined plasma devices called tokamaks--the design of a practical power plant remains an outstanding challenge. A key topic of current interest is microturbulence, which is believed to be responsible for the unacceptably large leakage of energy and particles out of the hot plasma core. Understanding and controlling this process is of utmost importance for operating current devices and designing future ones. In addressing such issues, the Gyrokinetic Toroidal Code (GTC) was developed to study the global influence of microturbulence on particle and energy confinement. It has been optimized on the IBM Blue Gene/L™ (BG/L) computer, achieving essentially linear scaling on more than 30,000 processors. A full simulation of unprecedented phase-space resolution was carried out with 32,768 processors on the BG/L supercomputer located at the IBM T. J. Watson Research Center, providing new insights on the influence of collisions on microturbulence.

The High Order Method Modeling Environment is a scalable, spectral-element-based prototype for the Community Atmospheric Model component of the Community Climate System Model. The 3D moist primitive equations are solved on the cubed sphere with a hybrid pressure η vertical coordinate using an Emanuel convective parametrization for moist processes. Semi-implicit time integration, based on a preconditioned conjugate gradient solver, circumvents the time step restrictions associated with gravity waves. Benchmarks for two standard tests problems at 10 km horizontal resolution have been run on Blue Gene/L. Results obtained on a 32-rack Blue Gene/L system (65,536 processors, 183.5-teraflop peak) show sustained performance of 8.0 teraflops on 32,768 processors for the moist Held–Suarez test problem in coprocessor mode and 11.3 teraflops on 32,768 processors for the aquaplanet test problem, running in virtual node mode.

Optimizing future supercomputing applications will depend on delivering the best performance for a given power budget. To determine the effect on efficiency of application-scaling parameters, this article analyzes system power and performance measurement results for real-world applications exploiting thread-and data-level parallelism on the blue gene/L system.

Applications on today's massively parallel supercomputers rely on performance analysis tools to guide them toward scalable performance on thousands of processors. However, conventional tools for parallel performance analysis have serious problems due to the large data volume that may be required. In this paper, we discuss the scalability issue for MPI performance analysis on Blue Gene/L, the world's fastest supercomputing platform. We present an experimental study of existing MPI performance tools that were ported to BG/L from other platforms. These tools can be classified into two categories: profiling tools that collect timing summaries, and tracing tools that collect a sequence of time-stamped events. Profiling tools produce small data volumes and can scale well, but tracing tools tend to scale poorly. The experimental study discusses the advantages and disadvantages for the tools in the two categories and will be helpful in the future performance tools design

A general method for optimizing problem layout on the Blue Gene((R))/L (BG/L) supercomputer is described. The method takes as input the communication matrix of an arbitrary problem as an array with entries C(i.j.), which represents the data communicated from domain i to domain j. Gh,en C(i, j), we implement a heuristic map that attempts to sequentially map a domain and its communication neighbors either to the same BG/L node or to nearneighbor nodes on the BG/L torus, while keeping the number of domains mapped to a BG/L node constant. We then generate a A,Markov, chain of maps using Monte Carlo simulation with free energy F = Sigma(ij) C(i,j) H(i, j), where H(i, j) is the smallest number of hops on the BG/L torus between domain i and domain j. For two large parallel applications, SAGE and UMT2000, the method was tested against the default Message Passing Interface rank order layout on up to 2,048 BG/L nodes. It produced maps that improved communication efficiency by up to 45%.

Good performance monitoring is the basis of modern performance analysis tools for application optimization. We are providing a variety of such performance analysis tools for the new Blue Gene®/L supercomputer. Those tools can be divided into two categories: single-node performance tools and multinode performance tools. From a single-node perspective, we provide standard interfaces and libraries, such as PAPI and libHPM, that provide access to the hardware performance counters for applications running on the Blue Gene/L compute nodes. From a multinode perspective, we focus on tools that analyze Message Passing Interface (MPI) behavior. Those tools work by first collecting message-passing trace data when a program runs. The trace data is then used by graphical interface tools that analyze the behavior of applications. Using the current prototype tools, we demonstrate their usefulness and applicability with case studies of application optimization.

Blue Gene/L represents a new way to build supercomputers, using a large number of low power processors, together with multiple integrated interconnection networks. Whether real applications can scale to tens of thousands of processors (on a machine like Blue Gene/L) has been an open question. In this paper, we describe early experience with several physics and material science applications on a 32,768 node Blue Gene/L system, which was installed recently at the Lawrence Livermore National Laboratory. Our study shows some problems in the applications and in the current software implementation, but overall, excellent scaling of these applications to 32K nodes on the current Blue Gene/L system. While there is clearly room for improvement, these results represent the first proof point that MPI applications can effectively scale to over ten thousand processors. They also validate the scalability of the hardware and software architecture of Blue Gene/L.