Tensor Cores (TCs) are specialized hardware units designed for efficient matrix multiplication and are widely utilized in deep learning workloads. However, their adoption in more irregular highperformance computing (HPC) applications remains limited. This paper presents a methodology for effectively integrating TCs into a representative HPC application: molecular docking with AutoDockGPU. The irregular computational patterns and strict accuracy requirements of this application pose significant challenges for TC utilization. To address these, we adopt a twofold strategy: (i) accelerating sum reduction operations using TCs, and (ii) applying state-of-the-art numerical error correction (EC) techniques to maintain accuracy. Experimental evaluations on NVIDIA A100, H100, and B200 GPUs show that our CUDA-based implementation consistently outperforms the baseline while preserving algorithmic accuracy.
Data movement bottlenecks have emerged as a dominant performance limiter in modern computing systems. At the same time, scientific detectors are generating unprecedented data volumes: X-ray detectors may soon produce terabytes per second and high-energy physics experiments demand bandwidth on the order of petabytes per second. Streaming compression offers a promising path to reducing data movement overhead, and hardware accelerators can further improve data throughput, while the exploration of system-level hardware compressors remains largely unexplored. This paper presents a preliminary study toward enabling hardware evaluation of streaming compressors. We design and implement a custom hardware accelerator for scientific data compression using modern hardware description languages, providing a complete end-to-end hardware acceleration system for CPU-based platforms. Our prototype features a multi-stage state machine, parallel element processing, and optimized data transfers, achieving 1.45x speedup over a software baseline with comparable quality, with 31% fewer cycles per element and 45% faster compression throughput.
We present an automated framework for online task scheduling on heterogeneous distributed systems, building on a modular parametric scheduler that enables dynamic scheduling decisions based on evolving execution states. Inspired by classical list-scheduling strategies such as HEFT and CPoP, our online scheduler simulates real-time task scheduling using only partial task graph knowledge. We evaluate our online scheduler variants against both their traditional offline baselines and a naive online strategy using a largescale benchmark suite of real-world scientific workflows. Experimental results across different estimation methods and compute-to-communication ratio (CCR) settings show that our adaptive online schedulers consistently outperform the naive approach, achieving performance within approximately 3-5% of an ideal offline scheduler that has full future knowledge (compared to the approximately 10% overhead for the naive baseline).
Foundation models, such as large language models, have become ubiquitous and are employed in many applications. These models are often adapted to a desired domain or downstream task through a fine-tuning stage. Cerebras, the wafer-scale AI accelerator, is a suitable platform for running foundation models. However, existing works utilizing sparse pre-training followed by dense fine-tuning result in dense models, failing to reduce the inference cost, measured in floating-point operations. This paper proposes Phoenix, a solution to enable sparse inference for downstream tasks on Cerebras. Phoenix features an innovative strategy that merges sparseweights with low-rank adapters to enable sparsity without losing accuracy, overcoming the limitations of previous approaches. Phoenix also leverages the Cerebras CS-2 wafer-scale engine to accelerate the inference process using unstructured sparsity methods. We evaluate multiple adaptation scenarios, models, and a range of sparsity levels to demonstrate the effectiveness of Phoenix. Phoenix maintains accuracy comparable to dense baselines even at 50-60% sparsity. Meanwhile, it achieves up to a near 2x reduction in FLOPs and a 1.7x improvement in inference throughput compared to a single NVIDIA A100 GPU during the inference stage.
Understanding the irregular, dynamic communication patterns in HPC applications at scale is critical when evaluating potential software optimizations and hardware architectures. Current systems monitor communication behavior for entire applications as exhaustive traces or general-purpose aggregated statistics. Generally, these approaches often do not scale well and the data gathered is often too generic or inflexible to make specific hardware/software optimizations. This paper describes a new, configurable, histogram-based approach to gathering scalable, high-fidelity monitoring information about HPC communication that we implemented in the Vernier communication monitoring system. This approach enables targeted collection of statistical data about annotated communication patterns for online or offline analysis, benchmarking, or network simulations. We assess these capabilities by collecting communication patterns from several production HPC applications at scale, showing that the resulting statistical representations accurately characterize the communication patterns in these applications, and can be used to provide new insights into communication patterns of complex HPC applications.
