As exascale systems increasingly rely on hardware overprovisioning to sustain throughput under strict power limits, accurate, fine-grained power prediction becomes critical. We present a machine learning framework leveraging lightweight, always-on NVIDIA DCGM telemetry from the Perlmutter super-computer to predict runtime GPU power consumption, designed with real-time deployment constraints in mind. Using a month-long production telemetry dataset for training and evaluation, we demonstrate that a generic, application-agnostic model achieves an average Root Mean Squared Error (RMSE) of 50 W. Specialized application-aware models for dominant workloads (e.g., Chroma, VASP) further reduce prediction error by up to 48% compared to generic models, while combining DCGM metrics with historical power data improves accuracy by up to 15.8% over power-only models with negligible inference latency. Our framework provides a scalable building block for dynamic power management systems that allow hardware-overprovisioned systems to reclaim performance otherwise lost to static power cans.

Power is a fundamental constraint as supercomputing advances to exascale. Efficient operation within strict power budgets requires application-aware power management based on a detailed understanding of application-level power behavior. This work analyzes the Energy Exascale Earth System Model (E3SM) atmosphere component, SCREAM, on Perlmutter (NERSC) and Frontier (OLCF). We characterize power variation across inputs, concurrency levels, and power caps, evaluate the energy impact of code optimizations, and attribute energy within the code using a newly developed GPU energy model. Results show that SCREAM’s peak power remains stable during its core execution phase and decreases gradually as concurrency increases. Power capping experiments reveal a performance–energy "sweet spot". On Perlmutter, limiting GPU power to 50% of thermal design power (TDP) achieves up to 15% energy savings with a 7% performance penalty. On Frontier, a 40% TDP cap yields up to 10% energy savings with less than 10% performance loss. Code optimizations reduce SCREAM energy by shortening run time without increasing power. Modeling reveals a critical insight: data movement accounts for approximately 70% of SCREAM’s GPU energy. This fundamentally shifts the optimization focus from FLOPS to data transfer reduction for this class of applications, offering the most impactful strategy for improving energy efficiency. This work establishes a foundation for practical, application-aware power management at exascale.

GPGPU-accelerated clusters and supercomputers are central to modern high-performance computing (HPC). Over the past decade, these systems continue to expand, and GPUs now expose a wide range of hardware counters that provide detailed views of performance and resource usage. Despite the potential of these counters, few studies have evaluated the insights they offer about real workloads at scale. In this work, we address this gap by analyzing previously underexplored GPU hardware counters collected via Lightweight Distributed Metric Service on Perlmutter, a leadership-class supercomputer. We quantify uneven work distribution across GPUs within a job and the steadiness of GPU activity over time, and we classify jobs as compute- or memory-bound using a roofline-based criterion. We then use these metrics to interpret job behavior in terms of practical workload characteristics to provide interpretable, job-level insights. Our findings can inform workload optimization and future HPC system design. For example, 81% of jobs are memory-bound, and memory-bound jobs tend to consume more energy than compute-bound jobs at comparable GPU-hours. Among jobs requesting 80 GB GPUs, 55% peak at 50% HBM capacity or less.

Quantum resources are increasingly integrated into high-performance computing (HPC) and cloud environments, but quantum high-performance computing (QHPC) software stacks remain isolated, often proprietary, full-stack solutions lacking common interfaces across runtime, resource management, orchestration, and execution layers. This paper analyzes nine production QHPC stacks and identifies common design patterns and emerging requirements, covering deployment models, application interaction patterns, SDK support, and readiness for fault-tolerant operation. The survey exposes consistent needs in runtime abstraction, resource management, interconnect semantics, and observability. Based on these findings, we propose the open quantum-HPC software ecosystem ( openQSE) reference architecture as a first step toward unifying the state-of-the-practice. openQSE defines a set of layer boundaries that allow different implementations to interoperate while preserving deployment flexibility, and is structured to support both current noisy intermediate-scale quantum (NISQ) workloads and future fault-tolerant quantum computing (FTQC) systems without changes to upper-layer application interfaces.

The National Energy Research Scientific Computing Center (NERSC), as the high-performance computing (HPC) facility for the Department of Energy's Office of Science, recognizes the essential role of quantum computing in its future mission. In this report, we analyze the NERSC workload and identify materials science, quantum chemistry, and high-energy physics as the science domains and application areas that stand to benefit most from quantum computers. These domains jointly make up over 50

GPGPU-accelerated clusters and supercomputers are central to modern high-performance computing (HPC). Over the past decade, these systems continue to expand, and GPUs now expose a wide range of hardware counters that provide detailed views of performance and resource usage. Despite the potential of these counters, few studies have evaluated the insights they offer about real workloads at scale. In this work, we address this gap by analyzing previously underexplored GPU hardware counters collected via Lightweight Distributed Metric Service on Perlmutter, a leadership-class supercomputer. We quantify uneven work distribution across GPUs within a job and the steadiness of GPU activity over time, and we classify jobs as compute- or memory-bound using a roofline-based criterion. We then use these metrics to interpret job behavior in terms of practical workload characteristics to provide interpretable, job-level insights. Our findings can inform workload optimization and future HPC system design. For example, 81

Maximizing supercomputing throughput within power and cooling limits is a key challenge for exascale systems, which are increasingly constrained by power rather than performance. Effective power management is essential. Whereas power capping has been well-known to increase energy efficiency and reduce energy costs, power variability has emerged as an orthogonal driving force on cost through service pricing models and increased electronic component wear out. This paper presents a performance-power-efficiency model that combines application performance, empirical power usage, power variability, and energy efficiency in a single methodology to enable optimization of HPC system operation. Using seven workloads and three microbenchmarks, we demonstrate the ability of our methodology to understand performance and energy efficiency through power capping on NVIDIA A100 GPUs and motivate future system design and execution policies. We show that power capping can reduce power spikes without sacrificing energy efficiency, and power capping can potentially improve power-constrained system throughput by 1.8× based on capped maximum node power and 2.5× based on peak node power usage.

As advances in energy-efficiency become the primary limiter to increases in power-constrained supercomputing and machine learning performance, it is imperative developers, architects, and practitioners understand how modern GPUs consume energy when running HPC and ML applications. Rather than opaque coarse-grained metrics, in this paper, we develop an extensible, microbenchmark-parameterized energy model capable of attributing application energy not only by functional unit (FPU, tensor core, integer ALU) and memory level (L1, L2, HBM), but can also differentiate control energy from datapath energy. We examine trends in energy per operation among four generations of GPUs and validate our results using supercomputing and ML/AI procurement workloads. Our insights and extrapolations can be used to drive the future of CMOS and memory technologies, computer architecture research, algorithmic innovation, optimizations for power-constrained and mobile environments, and data center operations.

Power is a critical limiting factor in supercomputing as systems scale to exascale levels. To advance scientific computing, supercomputers must operate efficiently under limited power budgets. Power-aware scheduling can help by enforcing power management strategies, but this requires a deep understanding of application power behavior, especially on modern GPU-centric supercomputers. This study examines the power behavior of VASP, a leading HPC application, on the Perlmutter A100 GPU system at NERSC. We explore how VASP’s power usage changes with various inputs and parallelism, and assess its response to power cappings. We find that VASP’s power usage varies significantly with different workloads, more so than with parallel concurrency. Additionally, power capping GPUs to 50% of their Thermal Design Power can be applied to most VASP workloads with less than a 10% performance loss. These findings shed light on the feasibility and effectiveness of power-aware scheduling based on application power profiles on HPC systems.

As next-generation experimental and observational instruments for scientific research are being deployed with higher resolutions and faster data capture rates, the fundamental demands of producing high-quality scientific throughput require portability and performance to meet the high productivity goals. Understanding such a workflow’s end-to-end performance on HPC systems is formidable work. In this paper, we address this challenge by introducing a Workflow Roofline model, which ties a workflow’s end-to-end performance with peak node- and system-performance constraints. We analyze four workflows: LCLS, a time-sensitive workflow that is bound by system external bandwidth; BerkeleyGW, a traditional HPC workflow that is bound by node-local performance; CosmoFlow, an AI workflow that is bound by the CPU preprocessing; and GPTune, an auto tuner that is bound by the data control flow. We demonstrate the ability of our methodology to understand various aspects of performance and performance bottlenecks on workflows and systems and motivate workflow optimizations.

HPC system architects routinely use many forms of application profiling and performance modeling to evaluate hardware and software performance trade-offs. However, the focus on individual applications can leave gaps in the understanding of total system utilization because it is impractical to collect profiles and models for every combination of application and input. In this paper, we use hardware activity metrics data gathered from thousands of GPUs on NERSC’s Perlmutter system to perform a roofline performance analysis of the full cross-section of a diverse scientific workload and provide quantitative empirical evidence for widely held beliefs that had previously been inferred from scattered analyses of individual applications. Specifically, we confirm the predominance of double-precision (FP64) floating point operations in scientific computing, responsible for two thirds of the total flop count; single-precision (FP32) accounts for another third while half-precision (FP16) operations are rare. Additionally, the arithmetic intensity for these operations are below the machine balance for 46% of samples and above it for 54%, which suggests near equal fractions of the workload are compute-bound and bandwidth-bound on Perlmutter’s GPUs. These results stand in contrast to hardware performance trends where artificial intelligence applications are driving processors to emphasize the performance of reduced-precision operations, and gains in memory bandwidth are not keeping pace with peak processing rates.

We develop a new model to evaluate the necessary classical computing and networking resources required to support a large-scale fault-tolerant quantum computer based on superconducting qubits and a surface code architecture. We focus specifically on quantum error decoding, which is the main classical computational task required to enable quantum error correction during runtime. Our model reveals that the quantum computer operates at a logical clock speed in the 100-10,000 Hz range, using state-of-the-art quantum error decoders. For a prototypical large-scale quantum chemistry computation, this translates to an overall runtime on the order of months, and this workload is estimated to generate syndrome data for error correction at a rate of 2–500 Gbps depending on whether data compression is used. We estimate the total computational processing power required for online error syndrome decoding equals about 1 petaflop. The results of our analysis show that current computing and networking technology can meet the requirements, in terms of bandwidth, latency, and compute, to support large-scale quantum computation. However, major technological challenges remain both for quantum and classical hardware, including scalable fabrication of high-quality qubits, scalable qubit control, and syndrome communication within a limited power budget.

Power has been a key constraint for supercomputers, and limitations on power become increasingly noticeable through the exascale era. Limited power availability pushes the facilities to operate under power constraints and develop power management methods, making it crucial to understand applications’ power consumption behavior and their performance under power constraints. In this study, we examine the power consumption of MILC, a widely used lattice quantum chromodynamics application, on the Perlmutter GPU system at NERSC. We analyze the power consumption of Generation and Spectrum applications of MILC using varying parallel concurrencies and input sizes. We then investigate the performance under GPU power caps and show that MILC is well-suited for GPU power capping. Up to 50% of GPU’s TDP can be applied to MILC jobs with less than 15% of performance decrease.

The rising power demands of supercomputers put high importance on understanding the underlying sources of power use. We compare a comprehensive set of power measurements covering six months from two supercomputers, the Cori and Perlmutter machines at the National Energy Research Scientific Computing Center (NERSC). We show that power usage varies considerably, and is always significantly below the peak provisioned power. Several factors cause this - the machine may not be fully utilized, applications' computational characteristics are not those which maximize power usage, and/or applications can be waiting on resources external to the node. Our analysis shows that while the power usage of applications in the same science domain is similar, the power usage of the same application run by different users is even more similar. As NERSC transitioned to GPU accelerated nodes, the peak power capabilities increased, but the production workload's power demands did not increase at the same rate, further decreasing the fraction of thermal design power (TDP) used. These results indicate that future machines could be power capped and over-provisioned and a metric different than thermal peak design is needed for future procurement, in alignment with the actual power needs of production workloads. These results suggest that with appropriate technologies, such as power-aware scheduling or dynamic power management, future HPC systems could be operated with power caps well below TDP, avoiding the high cost of over-provisioned infrastructure.

Generative AI, in particular large transformer models, are increasingly driving HPC system design in science and industry. We analyze performance characteristics of such transformer models and discuss their sensitivity to the transformer type, parallelization strategy, and HPC system features (accelerators and interconnects). We utilize a performance model that allows us to explore this complex design space and highlight its key components. We find that different transformer types demand different parallelism and system characteristics at different training regimes. Large Language Models are performant with 3D parallelism and amplify network needs only at pretraining scales with reduced dependence on accelerator capacity and bandwidth. On the other hand, long-sequence transformers, representative of scientific foundation models, place a more uniform dependence on network and capacity with necessary 4D parallelism. Our analysis emphasizes the need for closer performance modeling of different transformer types keeping system features in mind and demonstrates a path towards this.

We present a simple performance model to estimate the qubit-count and runtime associated with large-scale error-corrected quantum computations. Our estimates extrapolate current usage costs of quantum computers and show that computing the ground state of the 2D Hubbard model, which is widely believed to be an early candidate for practical quantum advantage, could start at a million dollars. Our model shows a clear cost advantage of up to four orders of magnitude for quantum processors based on superconducting technology compared to ion trap devices. Our analysis shows that usage costs, while substantial, will not necessarily block the road to practical quantum advantage. Furthermore, the combined effects of algorithmic improvements, more efficient error correction codes, and R&D cost amortization are likely to lead to orders of magnitude reductions in cost.

Power has become a key limiting factor in supercomputing. Understanding the power signatures of current production workloads is essential to address this limit and continue to advance scientific computing at scale. This paper analyzes the power characteristics of NERSC production workloads at the system and application levels. Our system-level analysis revealed a large gap between the average and peak power usage distribution, indicating a significant power swing from running various applications on the system. On the application level, we select four workflow benchmarks representing NERSC’s production workloads to analyze the power characteristics of applications and attempt to correlate the observed power timeline patterns with GPU performance metrics and application profiling data. We found different applications have distinct power usage patterns and widespread average and peak power usage. We discuss how these findings may help improve the current system’s operational power efficiency and the implications for future system procurement.

Power is a limiting factor for supercomputers limiting their scale and operation. Characterizing the power signatures of different application types can enable data centers to operate efficiently, even when power constrained. This paper investigates power profiles of diverse scientific applications, spanning both traditional simulations and modern machine learning (ML) running on the Perlmutter supercomputer at the National Energy Research Scientific Computing Center (NERSC). Our findings indicate that traditional simulations typically consume more power on average than ML workloads. Furthermore, ML applications exhibit periodic power fluctuations attributed to epoch transitions during training. Finally, we discuss the potential implications of the research insights toward automatic demand response (ADR) and considerations for designing future systems.