Current technologies allow for the sequencing of microbial communities directly from the environment without prior culturing. One of the major problems when analyzing a microbial sample is to taxonomically annotate its reads to identify the species it contains. In this study, we delve into an evaluation of how variations in the reference database impact classification performance. This aspect assumes paramount significance, particularly when considering the continuous evolution and expansion of the NCBI Reference Sequence database (RefSeq) over time. The aim of this study is to analyze the influence of RefSeq database size and composition on taxonomic identification performance using Kraken 2, a widely used taxonomic classification and profiling method. We found that resource-efficient Kraken 2 Lite (capped) databases, which can be run on standard laptops, generally exhibit lower classification accuracy compared to full reference databases that require high-performance computing infrastructure. The size of the RefSeq database and its growth affect the performance of the k-mer-based algorithms in addition to the computing resources need. We reported that changes in RefSeq reference database over time influenced the accuracy of metagenomic taxonomic classification, and training the classifier with more data, for example, new species, worsen the results especially on the most complex microbial communities.
IntroductionThe memory wall phenomenon—where advances in processor performance significantly outpace those in memory subsystems-poses a fundamental challenge for contemporary computing systems. In memory-bound applications, memory subsystem behavior dominates performance, yet existing analysis approaches present significant limitations: detailed microarchitectural simulators require days to weeks to simulate modest workloads; hardware performance counters provide only aggregate statistics that obscure temporal and spatial access patterns; and scaled simulation approaches face challenges in capturing contention effects, bandwidth saturation, and interference patterns that emerge at larger scales. These limitations reflect a processor-centric design philosophy—in both performance analysis tools and system co-design methodologies—that is increasingly misaligned with memorybound workloads, where a detailed understanding of memory access patterns, cache hierarchy interactions, and contention is critical for effective optimization.MethodsThis paper presents an integrated framework for memory-centric analysis that enables effective hardware-software co-design. We describe practical trace collection techniques, including hardware assisted processor tracing with minimal overhead and portable software-based instrumentation with statistical sampling. We present multi-perspective analysis methods that examine memory behavior from temporal, sequential, spatial, and relational viewpoints, revealing distinct optimization opportunities invisible in aggregate metrics. For example, a data structure switch from an open to a closed hash table in the miniVite graph application—guided by spatial anticipation metrics—improved hardware prefetcher utilization and delivered a 1.8 × runtime improvement, a benefit invisible to aggregate performance counters. We detail an architectural modeling framework that uses sampled traces with temporal interpolation and confidence-based filtering to evaluate cache and memory configurations.ResultsEvaluation on representative benchmarks demonstrates that this framework achieves practical accuracy (L2 cache errors of 2.64%, confidence-filtered L3 errors of 9.92%, bandwidth errors of 7.33%) while providing substantial speedup (26.8 × ) over cycle-accurate simulation, enabling rapid design space exploration.DiscussionWe demonstrate how this integrated framework enables systematic identification of both hardware optimizations (memory controller tuning, bank partitioning, NUMA configuration) and software optimizations (data layout restructuring, pre-fetching strategies, and memory-aware scheduling). Through this comprehensive treatment of the memory-centric analysis pipeline—from trace collection through architectural modeling to co-design application—we provide researchers and practitioners with practical techniques for addressing memory bottlenecks in contemporary computing systems.
Energy efficiency is a major challenge in High-Performance Computing (HPC) systems, impairing their scale, performance, and sustainability. Despite technological and research progress, there is still a lack of software methods to measure and assess the energy efficiency of computing codes at scale. This is also exacerbated by the emergence of newer ISAs in the HPC computing spectrum with non-unified interfaces for power and energy monitoring. In this work, we present CNTD_MERIC, which integrates two state-of-the-art energy monitoring and optimization libraries for HPC systems, COUNTDOWN and MERIC. COUNTDOWN is an energy-aware runtime system for MPI applications. MERIC is a platform-agnostic runtime system and energy measurement library that optimizes energy efficiency by adjusting hardware configurations. CNTD_MERIC combines the benefits of these two approaches with low overhead, resulting in a portable power management runtime system for MPI applications. We evaluated the integrated library on both ARM and x86 compute nodes in the production environment of the IT4Innovations supercomputing center (IT4I). The results show that CNTD_MERIC achieves similar performance to the original COUNTDOWN and MERIC implementations in terms of energy optimization and power/energy measurement, with negligible overheads within −5% to +3% compared to the original COUNTDOWN configurations. We also implemented CNTD_MERIC for multi-architecture (x86 and ARM) comparison between Intel Sapphire Rapids and A64FX processors. The results indicate that A64FX achieves significantly lower execution time, reduced energy-to-solution, and lower average power consumption (110–132 vs. 400–590 W), confirming its efficiency for energy-efficient HPC systems.
With the first exascale computing systems in production, tuning and scaling HPC applications to fully utilize the available hardware resources has become more important than ever. Thus, there is a strong need for software tools that assist application developers with this task. The Score-P instrumentation and measurement infrastructure plays a major role in filling this gap. Score-P is a community-driven, highly scalable tool suite for profiling and event tracing of massively parallel HPC application codes, and aimed to be easy to use. It provides measurement data via common data formats and runtime interfaces for a variety of complementary analysis tools developed by multiple institutions and companies, allowing users to gain insights into the communication, synchronization, input/output, and scaling behavior of their applications, pinpointing performance bottlenecks and their causes. In this article, we provide an overview of the current state of the Score-P infrastructure and its related tools ecosystem Cube, Extra-P, TAU, Scalasca, and Vampir. In particular, we detail Score-P's current design and architecture, both of which are highly flexible and extensible. Moreover, we describe how Score-P interacts with the analysis tools mentioned above and highlight the major extensions implemented over the past 10+ years to keep pace with the rapidly changing landscape of HPC hardware and parallel application programming interfaces. Furthermore, we discuss emerging challenges, particularly with respect to the ever-growing heterogeneity in both hardware and software, for collecting and analyzing performance data from applications running on future top-tier computing systems.
High-performance computing (HPC) applications face challenges in achieving scalability, with bottlenecks often discovered only late in the development cycle. Performance modeling offers a means to predict and understand scalability, but analytical approaches require deep expertise and are often impractical for large, complex codes. To address this, the Extra-P project provides a user-friendly tool for empirical performance modeling, enabling automated model generation from a small number of carefully selected experiments. This paper presents an overview of Extra-P, its underlying methodology—the Performance Model Normal Form (PMNF)—and its evolution into a mature tool for detecting and analyzing scalability issues. We discuss strategies to reduce experiment costs through parameter selection, sparse modeling, and Gaussian process regression, as well as techniques for mitigating the impact of noise using iterative refinement and deep learning. Furthermore, we highlight novel use cases, including segmented modeling and validation of user expectations, and of course demonstrate how Extra-P can uncover hidden bottlenecks in real-world applications such as HOMME or MPI libraries. Finally, we outline the software's architecture and future directions, emphasizing the potential for integration with AI-driven methods and adaptation to increasingly heterogeneous hardware.
With the growing complexity of I/O software stacks and the rise of data-intensive workloads, optimizing I/O performance is essential for enhancing overall system performance on HPC clusters. While many sophisticated I/O management approaches exist that try to alleviate I/O contention, they often rely on models that predict the future I/O behavior of applications. Yet, these models are often created from past execution runs and can be error-prone due to I/O variability. In this work, we propose an enhancement to an existing tool that leverages frequency-based techniques to characterize I/O phase. We explore methods to improve prediction accuracy by incorporating multiple frequency components. Furthermore, by coupling the wavelet transformation with the Fourier transformation, we enhance the precision of our predictions while maintaining a compact and efficient behavioral characterization. We demonstrate our approach using a deep learning benchmark executed on a production cluster.
Data is found everywhere, from health and human infrastructure to the surge of sensors and the proliferation of internet-connected devices. To meet this challenge, the data engineering field has expanded significantly in recent years in both research and industry. Traditionally, data engineering, Machine Learning, and AI workloads have been run on large clusters within data center environments, requiring substantial investment in hardware and maintenance. With the rise of the public cloud, it is now possible to run large applications across nodes without owning or maintaining hardware. Serverless functions such as AWS Lambda provide horizontal scaling and precise billing without the hassle of managing traditional cloud infrastructure. However, when processing large datasets, users often rely on external storage options that are significantly slower than direct communication typical of HPC clusters. We introduce Cylon, a high-performance distributed data frame solution that has shown promising results for data processing using Python. We describe how we took inspiration from the FMI library and designed a serverless communicator to tackle communication and performance issues associated with serverless functions. With our design, we demonstrate that the scaling efficiency of AWS Lambda achieves within 6.5% of serverful AWS (EC2) at 64 nodes, based on implementing direct communication via NAT Traversal TCP Hole Punching.
Binary Neural Networks (BNNs) perform competitively in practical image classification tasks. In particular, their accuracy is similar to that obtained by full-precision models tailored to edge devices. BNNs are amenable for devices with constrained resources, since they employ just one bit to store inputs and weights; hence, their storage requirements are low. Moreover, BNN computations utilize XNOR operations, which are implemented efficiently using simple hardware structures. However, supporting BNNs efficiently on edge devices is challenging since they perform many memory accesses to load weights, hindering their benefits. In BNNs, weights and inputs are stored using one bit, and several of them are packed together as a sequence of bits. In this work, the number of unique sequences representing a set of weights is observed to be low. In addition, a small group of unique sequences is seen to be used more frequently than others. Accordingly, this observation is exploited by using variable length encoding to represent the most commonly used sequences and decompressing them during inference. Commonly, BNNs employ floating-point values to store the weights for the output layer, aiming to obtain competitive accuracy. This further magnifies the storage requirements and significantly affects performance. In this work, it is observed that using traditional quantization schemes to quantize these weights negatively affects network accuracy. Then, it is shown that one can use a small number of bits to store most of the values in the output layer without negatively impacting accuracy by carefully quantizing the weights that fall outside a given range. Finally, an efficient quantization scheme for the output layer is proposed. A previously proposed FPGA accelerator is extended with the required hardware to implement the proposed schemes. The proposal is evaluated on a Pynq Z1 device. Our techniques reduce memory requirement by 1.5 × and improve performance by 2.2 × on average.
The diversity of processor architectures used for High-Performance Computing (HPC) applications has increased significantly over the last few years. This trend is expected to continue for different reasons, including the emergence of various instruction set extensions. Examples are the renewed interest in vector instructions like Arm's Scalable Vector Extension (SVE) or RISC-V's RVV. For application developers, research software developers, and performance engineers, the increased diversity and complexity of architectures have led to the following challenges: Limited access to these different processor architectures and more difficult root cause analysis in case of performance issues. To address these challenges, we propose leveraging the much-improved capabilities of processor simulators such as gem5. We enhanced this simulator with a performance analysis framework. We extend available performance counters and introduce new analysis capabilities to track the temporal behaviour of running applications. An algorithm has been implemented to link these statistics to specific regions. The resulting performance profiles allow for the identification of code regions with the potential for optimization. The focus is on observables to monitor quantities that are usually not directly accessible on real hardware. Different algorithms have been implemented to identify potential performance bottlenecks. The framework is evaluated for different types of HPC applications like the molecular-dynamics application GROMACS, Ligra, which implements the breadth-first search (BFS) algorithm, and a kernel from the Lattice QCD solver DD-αAMG.
High-performance computing (HPC) applications have traditionally relied on parallel file systems and file transfer services to manage data movement and storage. Alternative approaches have been proposed that use direct communications between application components, trading persistence and fault tolerance for speed. Event-driven architectures, as popularized in enterprise contexts, present a compelling middle ground, avoiding the performance cost and API constraints of parallel file systems while retaining persistence and offering impedance matching between application components. However, adapting streaming frameworks to HPC workloads requires addressing challenges unique to HPC systems. This paper investigates the potential for a streaming framework designed for HPC infrastructures and use cases. We introduce Mofka, a persistent event-streaming framework designed specifically for HPC environments. Mofka combines the capabilities of a traditional streaming service with optimizations tailored to the HPC context, such as support for massively multicore nodes, efficient scaling for large producer-consumer workflows, RDMA-enabled high-performance network communications, specialized network fabrics with multiple links per node, and efficient handling of large scientific data payloads. Built using the Mochi suite of HPC data service components, Mofka provides a lightweight, modular, and high-performance solution for persistent streaming in HPC systems. We present the architecture of Mofka and evaluate its performance against Kafka and Redpanda using benchmarks on diverse platforms, including Argonne's Polaris and Oak Ridge's Frontier supercomputers, showing up to 8× improvement in throughput in some scenarios. We then demonstrate its utility in several real-world applications: a tomographic reconstruction pipeline, a workflow for the discovery of metal-organic frameworks for carbon capture, and the instrumentation of Dask workflows for provenance tracking and performance analysis.
Modern high-performance computing (HPC) Input/Output (I/O) systems consist of stacked hard- and software layers that provide interfaces for data access. Depending on application needs, developers usually choose higher layers with richer semantics for the ease of use or lower layers for performance. Each I/O interface on a given stack consists of a set of operations and their syntactic definition, as well as a set of semantic properties. To properly function, high-level libraries such as Hierarchical Data Format version 5 (HDF5) need to map their semantics to lower-level Application Programming Interface (API) such as Portable Operating System Interface (POSIX). Lower-level storage backends provide different I/O semantics than the layers in the stack above while sometimes implementing the same interface. However, most I/O interfaces do not transport semantic information through their APIs. Ideally, no semantics of an I/O operation should be lost while passing through the I/O stack, allowing lower layers to optimize performance. Unfortunately, there is a lack of general definition and unified taxonomy of I/O semantics. Similarly, system-level APIs offer little support for passing semantics to underlying layers. Thus, passing semantic information between layers is currently not feasible. In this article, we systematically compare I/O interfaces by examining their semantics across the HPC I/O stack. Our primary goal is to provide a taxonomy and comparative analysis, not to propose a new I/O interface or implementation. We propose a general definition of I/O semantics and present a unified classification of I/O semantics based on the categories of concurrent access, persistency, consistency, spatiality, temporality, and mutability. This allows us to compare I/O interfaces in terms of their I/O semantics. We show that semantic information is lost while traveling through the storage stack, which often prevents the underlying storage backends from making the proper performance and consistency decisions. In other words, each layer acts like a semantic filter for the lower layers. We discuss how higher-level abstractions could propagate their semantics and assumptions down through the lower-levels of the I/O stack. As a possible mitigation, we discuss the conceptual design of semantics-aware interfaces, to illustrate how such interfaces might address semantic loss—though we do not propose a concrete new implementation.
Long-running scientific workflows, such as tomographic data analysis pipelines, are prone to a variety of failures, including hardware and network disruptions, as well as software errors. These failures can substantially degrade performance and increase turnaround times, particularly in large-scale, geographically distributed, and time-sensitive environments like synchrotron radiation facilities. In this work, we propose and evaluate resilience strategies aimed at mitigating the impact of failures in tomographic reconstruction workflows. Specifically, we introduce an asynchronous, non-blocking checkpointing mechanism and a dynamic load redistribution technique with lazy recovery, designed to enhance workflow reliability and minimize failure-induced overheads. These approaches facilitate progress preservation, balanced load distribution, and efficient recovery in error-prone environments. To evaluate their effectiveness, we implement a 3D tomographic reconstruction pipeline and deploy it across Argonne's leadership computing infrastructure and synchrotron facilities. Our results demonstrate that the proposed resilience techniques significantly reduce failure impact—by up to 500× —while maintaining negligible overhead (<3%).
IntroductionHigh-speed x-ray imaging experiments at synchrotron radiation facilities enable the acquisition of spatiotemporal measurements, reaching millions of frames per second. These high data acquisition rates are often prone to noisy measurements, or in the case of slower (but less noisy) rates, the loss of scientifically significant phenomena.MethodsWe develop a Shifted Window (SWIN)-based vision transformer to reconstruct high-resolution x-ray image sequences with high fidelity and at a high frame rate and evaluate the underlying algorithmic framework on a high-performance computing (HPC) system. We characterize model parameters that could affect the training scalability, quality of the reconstruction, and running time during the model inference stage, such as the batch size, number of input frames to the model, their composition in terms of low and high-resolution frames, and the model size and architecture.ResultsWith 3 subsequent low resolution (LR) frames and another 2 high resolution (HR) frames differing in the spatial and temporal resolutions by factors of 4 and 20, respectively, the proposed algorithm achieved an average peak signal-to-noise ratio of 37.40 dB and 35.60 dB.DiscussionFurther, the model was trained on the Argonne Leadership Computing Facility's Polaris HPC system using 40 Nvidia A100 GPUs, speeding up the end-to-end training time by about ~10 × compared to the training with beamline-local computing resources.
As next-generation scientific instruments and simulations generate ever larger datasets, there is a growing need for high-performance computing (HPC) techniques that can provide timely and accurate analysis. With artificial intelligence (AI) and hardware breakthroughs at the forefront in recent years, interest in using this technology to perform decision-making tasks with continuously evolving real-world datasets has increased. Digital twinning is one method in which virtual replicas of real-world objects are modeled, updated, and interpreted to perform such tasks. However, the interface between AI techniques, digital twins (DT), and HPC technologies has yet to be thoroughly investigated despite the natural synergies between them. This paper explores the interface between digital twins, scientific computing, and machine learning (ML) by presenting a consistent definition for the digital twin, performing a systematic analysis of the literature to build a taxonomy of ML-enhanced digital twins, and discussing case studies from various scientific domains. We identify several promising future research directions, including hybrid assimilation frameworks and physics-informed techniques for improved accuracy. Through this comprehensive analysis, we aim to highlight both the current state-of-the-art and critical paths forward in this rapidly evolving field.
X-ray crystallography reconstruction, which transforms discrete X-ray diffraction patterns into three-dimensional molecular structures, relies critically on accurate Bragg peak finding for structure determination. As X-ray free electron laser (XFEL) facilities advance toward MHz data rates (1 million images per second), traditional peak finding algorithms that require manual parameter tuning or exhaustive grid searches across multiple experiments become increasingly impractical. While deep learning approaches offer promising solutions, their deployment in high-throughput environments presents significant challenges in automated dataset labeling, model scalability, edge deployment efficiency, and distributed inference capabilities. We present an end-to-end deep learning pipeline with three key components: (1) a data engine that combines traditional algorithms with our peak matching algorithm to generate high-quality training data at scale, (2) a modular architecture that scales from a few million to hundreds of million parameters, enabling us to train large expert-level models offline while deploying smaller, distilled models at the edge, and (3) a decoupled producer-consumer architecture that separates specialized data source layer from model inference, enabling flexible deployment across diverse computing environments. Using this integrated approach, our pipeline achieves accuracy comparable to traditional methods tuned by human experts while eliminating the need for experiment-specific parameter tuning. Although current throughput requires optimization for MHz facilities, our system's scalable architecture and demonstrated model compression capabilities provide a foundation for future high-throughput XFEL deployments.
We present the implementation of a specialized version of our previously published unified embedding model, SpeckleNN, for real-time speckle pattern classification in X-ray Single-Particle Imaging (SPI), using the SLAC Neural Network Library (SNL) on an FPGA platform. This hardware realization transitions SpeckleNN from a prototypic model into a practical edge solution, optimized for running inference near the detector in high-throughput X-ray free-electron laser (XFEL) facilities, such as those found at the Linac Coherent Light Source (LCLS). To address the resource constraints inherent in FPGAs, we developed a more specialized version of SpeckleNN. The original model, which was designed for broader classification across multiple biological samples, comprised ~5.6 million parameters. The new implementation, while reducing the parameter count to 64.6K (a 98.8% reduction), focuses on maintaining the model's essential functionality for real-time operation, achieving an accuracy of 90%. Furthermore, we compressed the latent space from 128 to 50 dimensions. This implementation was demonstrated on the KCU1500 FPGA board, utilizing 71% of available DSPs, 75% of LUTs, and 48% of FFs, with an average power consumption of 9.4W according to the Vivado post-implementation report. The FPGA performed inference on a single image with a latency of 45.015 microseconds at a 200 MHz clock rate. In comparison, running the same inference on an NVIDIA A100 GPU resulted in an average power consumption of ~73W and an image processing latency of around 400 microseconds. Our FPGA-accelerated version of SpeckleNN demonstrated significant improvements, achieving an 8.9 × speedup and a 7.8 × reduction in power consumption compared to the GPU implementation. Key advancements include model specialization and dynamic weight loading through SNL, which eliminates the need for time-consuming FPGA design re-synthesis, allowing fast and continuous deployment of models (re)trained online. These innovations enable real-time adaptive classification and efficient vetoing of speckle patterns, making SpeckleNN more suited for deployment in XFEL facilities. This implementation has the potential to significantly accelerate SPI experiments and enhance adaptability to evolving experimental conditions.
The authors explore performance scalability of the open-source thermal-fluids code, NekRS, on the U.S. Department of Energy's leadership computers, Crusher, Frontier, Summit, Perlmutter, and Polaris. Particular attention is given to analyzing performance and time-to-solution at the strong-scale limit for a target efficiency of 80%, which is typical for production runs on the DOE's high-performance computing systems. Several examples of anomalous behavior are also discussed and analyzed.
