Training Graph Neural Networks (GNNs) on large graphs is typically done on complex, distributed, or Out-of-Core (OOC) systems due to memory pressure. But we challenge this assumption as training typically requires access to only a small subset of node features. As a result, training on many large graphs exhibits relatively small working sets that can fit on a modest GPU. We present a simple single-GPU framework that pre-computes computation graphs and feature subsets to eliminate CPU–GPU data transfers and runtime overheads. Our approach achieves reductions in epoch time of up to 45× over the distributed DistDGL framework and ≈ 2× over the OOC DiskGNN framework, demonstrating that a modern GPU is all you need for efficient large-scale GNN training.
Accurate prediction of parallel application performance in HPC systems is essential for efficient resource allocation and system design. Classical performance models estimate of speedup based on theoretical assumptions, but their applicability is limited by parameter estimation, data acquisition, and real-world system issues such as latency and network congestion. This paper describes performance prediction using classical performance models boosted by a trainable machine learning framework. Domain-informed machine-learning models estimate the overhead of an application for a given problem size and resource configuration as a coefficient of the estimated speedup provided by performance laws. We evaluate this approach on two HPC mini-applications and two full applications with varying patterns of computation and communication and also evaluate the prediction accuracy on runs with varying processors-per-node configurations. Our results show that this method significantly improves the accuracy of performance predictions over standard analytical models and black-box regressors, while remaining robust even with limited training data.
The deployment of deep neural networks on edge devices remains challenging due to their computational demands and memory requirements. We introduce a novel compression framework that integrates Layer-wise Relevance Propagation (LRP) with high-dimensional clustering to simultaneously identify and eliminate both unimportant and redundant neurons. Unlike conventional magnitude or gradient-based pruning methods, our approach computes LRP scores across all network layers to capture functional dependencies, performs global k-means clustering to group functionally similar neurons, and implements parameter sharing through representative neurons or synthetic centroids. We provide empirical analysis demonstrating how relevance conservation principles yield more reliable importance estimates than traditional methods. Extensive experiments on MNIST and CIFAR-10 datasets using fully connected networks, ResNet-9, and ResNet-18 architectures demonstrate compression ratios up to 16x with minimal accuracy degradation (less than 2% in most cases), outperforming pruning methods including SNIP, GraSP, and Fisher pruning. Edge deployment on platforms ranging from NVIDIA Jetson to Raspberry Pi shows consistent performance and reduced inference latency, validating our method's practical applicability for resource-constrained environments.
We introduce a novel, efficient computational method, ZEUS, for numerical optimization, and provide an open-source implementation. It has four key ingredients: (1) particle swarm optimization (PSO), (2) the use of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method, (3) automatic differentiation (AD), and (4) GPUs. Our approach addresses the computational challenges inherent in high-dimensional, non-convex optimization problems. In the first phase of the algorithm, we get a potentially good set of starting points using PSO. Thereafter, we run BFGS independently in parallel from these starting points. BFGS is one of the bestperforming algorithms for numerical optimization. However, it requires the gradient of the function being optimized. ZEUS integrates automatic differentiation into BFGS thus avoiding the need for the user to calculate derivatives explicitly. The use of GPUs allows ZEUS to speed up the calculations substantially. We carry out systematic studies to explore the trade-offs between the number of PSO iterations taken, starting points, and BFGS iteration depth. We show that a handful of iterations of PSO can improve global convergence when combined with BFGS. We also present performance studies using common test functions. The source code can be found at https://github.com/fnal-numerics/global-optimizer-gpu.
Efficient execution of range queries and exact queries on metadata is critical for many scientific workflows on large-scale high-performance computing (HPC) systems. Range queries in a distributed setting pose a challenge, as typical partitioning methods such as hash-based, list-based, and range-based partitioning do not work efficiently. There are also challenges in balancing the workload using locality-sensitive partitioning methods, which are capable of routing range queries to a subset of nodes to achieve efficiency. In this paper, we study prefix-hash tree and range-hash partitioning methods that fit the requirements of efficient range queries in distributed HPC systems. We also proposed a novel load-balancing method inspired by the Gossip protocol, which has similar complexity for most operations while keeping the coefficient of variation (CV) less than 7% across diverse metadata workload distributions, i.e., normal, uniform, and exponential. Our proposed methods outperform the commonly used Key-Value store, RocksDB, for exact queries by up to 15x and for range queries by up to 120,000x.
