Digital Twins exhibit heterogeneous spatial accuracy requirements across simulation domains. Spatial error–driven Level of Detail schemes exist for rasterization but are not directly applicable to Bounding Volume Hierarchy–traversal algorithms, such as ray tracing or collision detection. This work introduces a unified detail hierarchy that supports both algorithmic paradigms. We integrate clustered triangle mesh representations (meshlets) with multi-attribute mesh simplification while tracking approximation error per cluster. We present a specialized Quadric Error Metric that accurately captures the spatial error required for Digital Twin simulations. Additionally, we propose a unified clustering method that balances the spatial requirements for low Surface Area Heuristic in Bounding Volume Hierarchy traversal with the topological and fullness constraints necessary for rasterization, enabling a single cluster representation. We further introduce a method to integrate clusters from all Levels of Detail with spatial error into a single Bounding Volume Hierarchy by alternating construction priorities between spatial error and Surface Area Heuristic at consecutive levels. This mitigates excessive overlap from stacked clusters and improves pruning during traversal. The method is highly parallelizable and exhibits scalability on both CPUs and GPUs. Experiments demonstrate substantial geometry pruning within spatial error thresholds, yielding significant computational savings while maintaining accuracy requirements.
This article introduces parallALT—a fully spectral spherical harmonic transform library. Developed for modern massively parallel accelerators, such as GPUs, parallALT combines close to machine-precision accuracy with exceptional performance. This article demonstrates that our fully spectral algorithm has \(O(\sqrt {L})\) (with L being the spherical harmonic degree) round trip accuracy scaling compared to O(L) scaling of the conventional quadrature approach. We also show that parallALT implementation for high L ∈ [512, 16384] is up to three times faster on an Nvidia H100 GPU than the most efficient open source quadrature implementation, SHTns. Developed with the help of the in-house cross-platform runtime code generation platform, PfSolve and VkFFT libraries, parallALT will be released as an open source library in the future.
This article presents a high performance fully in-GPU inductive logic learning algorithm. The proposed algorithm performs all inductive logic learning activities such as hypothesis search and hypothesis evaluation, fully in-GPU with zero CPU involvement. The core contribution of this work, is a high performance fully in-GPU ILP (inductive logic programming) algorithm in description logic (DL), that also supports parallel hypothesis search. The proposed learner addresses the ILP research gap of non-existing GPU-based parallel hypothesis search approaches that target more expressive logics like FOL (first order logic) and DL. Within the proposed learner, we also propose a set of high performance GPU-based hypothesis evaluation algorithms. In addition, the proposed learner is based on the DL-Learner[ 11 ] (the state-of-the-art in DL-based ILPs), with which the proposed learner incorporates many of the DL-Learner’s search strategies – like deterministic ordering of hypotheses operands, to reduce the size of the search space. According to experimental results on four datasets, the proposed in-GPU learner is always faster than the DL-Learner on all datasets; for example, in the IMDB dataset, the proposed learner is faster by up to ~ 18 folds than the DL-Learner (represented using SPILDL at BeamWidth = 1). When considering the average of all hypothesis search beam widths, the proposed learner is always faster than SPILDL[ 6 ] on all datasets; SPILDL is our high performance CPU-based parallel learner in DL. In the Dunnhumby Retail dataset, on average, the proposed learner is faster than SPILDL by ~ 6.55 folds. In terms of worst case scenario, it is demonstrated through parallel search at certain beam width values. For example, at BeamWidth = 8 in the trains dataset, SPILDL is faster than the proposed learner by ~ 2.4 folds.
Deep neural networks (DNNs) have become foundational to modern applications, yet their substantial computational and memory demands pose major obstacles to energy-efficient inference. Moreover, the rapidly expanding parameter footprint and structural diversity further amplify data movement, leading to substantial energy consumption and latency overheads. To address these issues, we propose a novel accelerator, S-TRAC , that dynamically adjusts sparsity t h r esholds through a lgorithm-hardware c o-design to enable efficient DNN inference. At the algorithm level, we employ a static sparse-dense storage format and a dynamic bit-processing scheme to skip non-contributing bits without sacrificing weight precision. At the hardware level, we introduce a column-wise processing-element array with LUT-based shift-accumulate multiplication and a global partial-sum accumulator to sustain energy-efficient execution. To support the proposed algorithm-hardware co-design, we propose a RISC-V extension that coordinates the read, arrangement, multiplication, accumulation, and write stages to support end-to-end accelerator execution. Experimental results show that S-TRAC increases effective sparsity by an average factor of 8.4 × across the evaluated DNN models, enabling substantial memory savings. S-TRAC design achieves 11.16 × energy efficiency and 37.03 × hardware efficiency improvements over state-of-the-art solutions.
In this study, approaches to the development of interactive Java algorithms intended for dynamic visualization of parallel computational threads were considered. The proposed interactive Java algorithms give an opportunity to create visual graphical representations of parallel processes, their interactions, and data distribution. Within the framework of the research the key approaches to visualization of parallel computational flows are analyzed, the peculiarities of the applied interactive components are estimated and the methods of integration with existing monitoring and debugging systems are considered. With the help of configurable visualization tools, developers and researchers can observe the evolution of computational threads, evaluate the performance of systems and timely react to changes in the structure of parallel tasks. Implementing algorithms on the Java platform ensures portability, broad applicability, and integration with existing frameworks for high-performance computing. The use of dynamic data structures, thread-safe collections and parallelism mechanisms implemented in the language and standard libraries allows efficiently processing large amounts of data in real time. In addition, Java Virtual Machine provides profiling tools that can be directly applied to optimize the visualized processes.
We revisit the classical non-clairvoyant problem of scheduling a set of n jobs on a set of m parallel identical machines where the processing time of a job is not known until the job finishes. Our objective is the minimization of the makespan, i.e., the date at which the last job terminates its execution. We adopt the framework of learning-augmented algorithms and we study the question of whether (possibly erroneous) predictions may help design algorithms with a competitive ratio which is good when the prediction is accurate (consistency), deteriorates gradually with respect to the prediction error (smoothness), and not too bad and bounded when the prediction is arbitrarily bad (robustness). We first consider the non-preemptive case and we devise lower bounds, as a function of the error of the prediction, for any deterministic learning-augmented algorithm. Then we analyze a variant of the Longest Processing Time first algorithm (with and without release dates) and we prove that it is consistent, smooth, and robust. Furthermore, we study the preemptive case and we provide lower bounds for any deterministic algorithm with predictions as a function of the prediction error. Finally, we introduce a variant of the classical Round Robin algorithm, the Predicted Proportional Round Robin algorithm, which we prove to be consistent, smooth, and robust.
In this article, we focus on the communication costs of three symmetric matrix computations: (i) multiplying a matrix with its transpose, known as a symmetric rank-k update (SYRK) (ii) adding the result of the multiplication of a matrix with the transpose of another matrix and the transpose of that result, known as a symmetric rank-2k update (SYR2K) (iii) performing matrix multiplication with a symmetric input matrix (SYMM). All three computations appear in the Level 3 Basic Linear Algebra Subroutines (BLAS) and have wide use in applications involving symmetric matrices. We establish communication lower bounds for these kernels using sequential and distributed-memory parallel computational models, and we show that our bounds are tight by presenting communication-optimal algorithms for each setting. Our lower bound proofs rely on applying a geometric inequality for symmetric computations and analytically solving constrained nonlinear optimization problems. The symmetric matrix and its corresponding computations are accessed and performed according to a triangular block partitioning scheme in the optimal algorithms.
Priority queues are used in a wide range of applications, including prioritized online scheduling, discrete event simulation, and greedy algorithms. In parallel settings, classical priority queues often become a severe bottleneck, resulting in low throughput. Consequently, there has been significant interest in concurrent priority queues with relaxed semantics. In this article, we present the MultiQueue , a highly scalable and flexible relaxed priority queue. Its design is based on leveraging multiple internal priority queues using the seemingly paradoxical technique of “wait-free locking”. The practical performance of the MultiQueue is further enhanced by element buffering, batched operations on the internal queues, and cache-optimized access patterns. We evaluate the quality-throughput tradeoff of the MultiQueue against state-of-the-art competitors. The quality is measured using two complementary metrics: rank error (the distance of a deleted element to the best element), and delay (the number of elements with lower priority deleted before a given element). Extensive experiments on both micro-benchmarks and benchmarks based on real-world applications (shortest-path and branch-and-bound) show that the MultiQueue consistently outperforms its competitors. Additionally, its design allows users to easily tune the balance between throughput and quality to meet specific application requirements, and we believe that “wait-free locking” may be of broader interest for converting sequential data structures into high-performance, relaxed concurrent ones.
The reduce-scatter collective operation in which p processors in a network of processors collectively reduce p input vectors into a result vector which is partitioned into p blocks over the processors is important both in its own right and as building block for other collective operations. We present a surprisingly simple, but non-trivial algorithm for solving this problem optimally in ⌈log 2p⌉ communication rounds with each processor sending, receiving and reducing exactly p − 1 blocks of vector elements, provided that the binary reduction operator is commutative. We combine this with a similarly simple, well-known allgather algorithm to get a volume optimal algorithm for the allreduce collective operation where the result vector is replicated on all processors. The communication pattern is a simple, ⌈log 2p⌉-regular, circulant graph also used elsewhere. A simple requirement on the skip sequence of the circulant graph gives freedom in the choice of communication pattern which may be valuable for practical implementation. The algorithms can readily be implemented and used for the collective operations MPI_Reduce_scatter_block, MPI_Reduce_scatter_block and MPI_Allreduce as specified in the MPI standard. For completeness, we also give a ⌈log 2p⌉ communication round optimal allreduce algorithm that can be used to implement MPI_Allreduce and is suited for smaller input vectors. Finally, we observe that the reduce-scatter algorithm can be used as a template for round-optimal all-to-all communication with smaller communication volume, for some p, than the well-known algorithm by Bruck et al. (1997) to implement the collective MPI_Alltoall operation. We provide experimental evidence on a small cluster that the algorithms can perform better than or comparably to other implemented MPI library algorithms.
Constructing k-nearest neighbor (kNN) graphs is a fundamental component in many machine learning and scientific computing applications. Despite its prevalence, efficiently building all-nearest-neighbor graphs at scale on distributed heterogeneous HPC systems remains challenging, especially for large sparse non-integer datasets. We introduce optimizations for algorithms based on forests of random projection trees. Our novel GPU kernels for batched, within leaf, exact searches achieve 1.18× speedup over sparse reference kernels with less peak memory, and up to 19× speedup over CPU for memory-intensive problems. Our library, PyRKNN, implements distributed randomized projection forests for approximate kNN search. Optimizations to reduce and hide communication overhead allow us to achieve 5× speedup, in per iteration performance, relative to GOFMM (another projection tree, MPI-based kNN library), for a 64M 128d dataset on 1,024 processes. On a single-node we achieve speedup over FAISS-GPU for dense datasets and up to 10× speedup over CPU-only libraries. PyRKNN uniquely supports distributed memory kNN graph construction for both dense and sparse coordinates on CPU and GPU accelerators.
In this paper, we present PfSolve — a new, performant, cross-platform, and open-source implementation of tridiagonal and bidiagonal matrix solvers for the GPU architecture. Released as a stand-alone library, PfSolve can solve systems of arbitrary size that fit into the memory of a single GPU with a potential extension to multi-GPU support in the future. The code works in single, double, and double-double emulation of quad precision using only \(0.1\%\) of the original system size as additional memory. PfSolve is based on the in-house implementation of the Parallel Thomas algorithm optimized for GPU execution by using warp-level instructions and occupancy optimizations, which are discussed in detail in the paper. This work also presents an accuracy analysis of the Parallel Thomas algorithm for tridiagonal matrices with various dominance factors (approximately, the ratio of the off-diagonal to diagonal terms) and demonstrates that PfSolve achieves a considerable speedup over vendor solutions on modern HPC GPUs like Nvidia H100 and AMD MI210. The source code for PfSolve is available on GitHub.
Tensor transposition is a fundamental operation in tensor calculations with various applications. However, a naive implementation that copies each element from the source tensor to the transposed position in the target tensor requires double space, making it unsuitable for large-scale tensors on memory-limited accelerators, like Graphic Processing Units (GPUs). In this article, we propose an algorithm and its implementation, called EITHOT, for In-place Transposition of High Order Tensors on GPUs, which requires only 5% additional memory at most for large high order tensors. To achieve this, EITHOT uses a newly proposed method, called permutation decomposition, to factorize a transposition of a high-order tensor into a sequence of low-order tensor transpositions. Then, based on the estimated extra memory requirements, EITHOT divides a large tensor into smaller tensors and transposes each smaller tensor separately. Finally, the transposed smaller tensors are combined to form the desired result. The GPU implementation optimizes memory access performance using the cooperative groups programming model. Our experiments demonstrate that EITHOT delivers competitive performance compared to the state-of-the-art out-of-place GPU implementations. Furthermore, EITHOT can handle nearly double the size of tensors compared to out-of-place methods, making it suitable for various transpositions of N-order tensors.
Massively parallel systems, such as Graphics Processing Units (GPUs), play an increasingly crucial role in today’s data-intensive computing. The unique challenges associated with developing system software for massively parallel hardware to support numerous parallel threads efficiently are of paramount importance. One such challenge is the design of a dynamic memory allocator to allocate memory at runtime. Traditionally, memory allocators have relied on maintaining a global data structure, such as a queue of free pages. However, in the context of massively parallel systems, accessing such global data structures can quickly become a bottleneck even with multiple queues in place. This paper presents a novel approach to dynamic memory allocation that eliminates the need for a centralized data structure. Our proposed approach revolves around letting threads employ random search procedures to locate free pages. Through mathematical proofs and extensive experiments, we demonstrate that the basic random search design achieves lower latency than the best-known existing solution, Ouroboros, in most situations. Furthermore, we develop more advanced techniques and algorithms to tackle the challenge of warp divergence and further enhance performance when free memory is limited. Building upon these advancements, our mathematical proofs and experimental results affirm that these advanced designs can yield an order of magnitude improvement over the basic design and consistently outperform the state-of-the-art by up to two orders of magnitude. To illustrate the practical implications of our work, we integrate our memory management techniques into two GPU algorithms: a hash join and a group-by. Both case studies provide compelling evidence of our approach’s pronounced performance gains.
We give optimally fast O(log p) time (per processor) algorithms for computing round-optimal broadcast schedules for message-passing parallel computing systems. This affirmatively answers difficult questions posed in a SPAA 2022 BA and a CLUSTER 2022 paper. We observe that the computed schedules and circulant communication graph can likewise be used for reduction, all-broadcast and all-reduction as well, leading to new, round-optimal algorithms for these problems. These observations affirmatively answer open questions posed in a CLUSTER 2023 paper. The problem is to broadcast n indivisible blocks of data from a given root processor to all other processors in a (subgraph of a) fully connected network of p processors with fully bidirectional, simultaneous send-receive, one-ported communication capabilities. In this model, \(n-1+\lceil \log _2 p\rceil\) send-receive communication rounds are required. Our new algorithms work for any number of processors p, and compute for each processor in the network receive and send schedules each of size \(\lceil \log _2 p\rceil\) that determine uniquely in O(1) time for each communication round the new block that the processor will receive, and the already received block it has to send. Schedule computations are done independently per processor without communication. The broadcast communication subgraph is an easily computable, directed, \(\lceil \log _2 p\rceil\)-regular circulant graph also used elsewhere. We show how the schedule computations can be done in optimal time and space of O(log p), improving significantly over previous results of \(O(p\log ^2 p)\) and O(log3 p), respectively, for any number of processors. The schedule computation and broadcast algorithms are simple to implement, but correctness and complexity are not obvious. By dividing a given input of m Bytes into n indivisible blocks, the schedules are used for new, best possible pipelined implementations of the MPI (Message-Passing Interface) collectives MPI_Bcast, MPI_Reduce, MPI_Allgatherv, and MPI_Reduce_scatter. Indicative, experimental results are given. The schedules can also be used for replicating large arrays on an EREW PRAM.
In this article, we focus on the communication costs of three symmetric matrix computations: (i) multiplying a matrix with its transpose, known as a symmetric rank-k update (SYRK) (ii) adding the result of the multiplication of a matrix with the transpose of another matrix and the transpose of that result, known as a symmetric rank-2k update (SYR2K) (iii) performing matrix multiplication with a symmetric input matrix (SYMM). All three computations appear in the Level 3 Basic Linear Algebra Subroutines (BLAS) and have wide use in applications involving symmetric matrices. We establish communication lower bounds for these kernels using sequential and distributed-memory parallel computational models, and we show that our bounds are tight by presenting communication-optimal algorithms for each setting. Our lower bound proofs rely on applying a geometric inequality for symmetric computations and analytically solving constrained nonlinear optimization problems. The symmetric matrix and its corresponding computations are accessed and performed according to a triangular block partitioning scheme in the optimal algorithms.
In non-clairvoyant scheduling, the task is to find an online strategy for scheduling jobs with a priori unknown processing requirements with the objective to minimize the total (weighted) completion time. We revisit this well-studied problem in a recently popular learning-augmented setting that integrates (untrusted) predictions in online algorithm design. While previous works used predictions on processing requirements, we propose a new prediction model, which provides a relative order of jobs which could be seen as predicting algorithmic actions rather than parts of the unknown input. We show that these predictions have desired properties, admit a natural error measure as well as algorithms with strong performance guarantees and that they are learnable in both, theory and practice. We generalize the algorithmic framework proposed in the seminal paper by Kumar et al. (NeurIPS'18) and present the first learning-augmented scheduling results for weighted jobs and unrelated machines. We demonstrate in empirical experiments the practicability and superior performance compared to the previously suggested single-machine algorithms.
Clustering is a fundamental task in machine learning. One of the most successful and broadly used algorithms is DBSCAN, a density-based clustering algorithm. DBSCAN requires ϵ-nearest neighbor graphs of the input dataset, which are computed with range-search algorithms and spatial data structures like KD-trees. Despite many efforts to design scalable implementations for DBSCAN, existing work is limited to low-dimensional datasets, as constructing ϵ-nearest neighbor graphs can be expensive in high-dimensions. This paper introduces a modified DBSCAN, using k-nearest neighbor (kNN) graphs to improve efficiency. We outline conditions for kNN-DBSCAN to match DBSCAN’s results and present a parallel implementation using OpenMP and MPI for shared and distributed memory systems. Testing on datasets up to 32 dimensions, we achieve remarkable scalability. Our implementation clusters one billion 3D points in under one second on 28K cores at TACC’s Frontera system. In a larger run, we cluster 65 billion points in 20 dimensions in under 40 seconds using 114,688 cores. Our method is up to 37 × faster than state-of-the-art parallel DBSCAN on a 20-dimensional dataset with 4 million points. Code is available at https://github.com/ut-padas/knndbscan.
We investigate routing on networks modeled as multiple access channels, when packets are injected continually. An energy cap is a component of the system, understood as a bound on the number of stations that can be switched on simultaneously. Each packet is injected into some station and needs to be delivered to its destination station via the channel. A station has to be switched on in order to receive a packet when it is heard on the channel. Each station manages when it is switched on and off by way of a programmable wake-up mechanism, which is scheduled by a routing algorithm. Packet injection is governed by adversarial models that determine upper bounds on injection rates and burstiness. We develop deterministic distributed routing algorithms and assess their performance in the worst-case sense. An algorithm knows the number of stations but does not know the adversary. One of the algorithms maintains bounded queues for the maximum injection rate 1 subject only to the energy cap 3. This energy cap is provably optimal, in that obtaining the same throughput with the energy cap 2 is impossible. We give algorithms subject to the minimum energy cap 2 that have latency polynomial in the total number of stations~n for each fixed adversary of injection rate less than 1. An algorithm is k-energy-oblivious if at most k stations are switched on in a round and for each station the rounds when it will be switched on are determined in advance. We give a k-energy-oblivious algorithm that has packet delay O(n) for adversaries of injection rates less than (k-1)/(n-1), and show that there is no k-energy-oblivious stable algorithm against adversaries with injection rates greater than k/n. An algorithm routes directly when each packet makes only one hop from the station into which it is injected straight to its destination. We give a k-energy-oblivious algorithm routing directly, which has latency O(n^2/k) for adversaries of sufficiently small injection rates that are O(k^2/n^2). We develop a k-energy-oblivious algorithm routing directly, which is stable for injection rate k(k-1)/n(n-1), and show that no k-energy-oblivious algorithm routing directly can be stable against adversaries with injection rates greater than k(k-1)/n(n-1).
The Longest Common Subsequence (LCS) problem is a well-known and studied problem in computer science and bioinformatics. It consists in finding the longest subsequence that is common to two or more given sequences. In this article, we address the problem of finding all LCS for the Sequential Substring ConstrainedLCS (SSCLCS) problem, called the Multiple SSCLCS problem. To solve this problem, we first propose a dominant point-based sequential algorithm, designed on a new Leveled Direct Acyclic Graph (DAG) that gives the correct evaluation order of subproblems to avoid redundancy due to overlap. Depending on whether the constraints may overlap or not, it requires O ( S |Sigma| K + 1 + r + n |Sigma|) and O ( S |Sigma| K + 1 + n |Sigma|) time with O(Max_level+n|Sigma|) space. S is the number of partial SSCLCS in a node, K is the number of DAG levels, n is the length of sequences, r is the total length of constraints, Max_level is the number of nodes in the largest level of the DAG, and |Sigma| is the length of the alphabet. Then, we derive a coarse-grained multicomputer parallel solution requiring O ( S | Sigma| K +1 + r + n |Sigma | ) and O ( S | Sigma|K+1+n |Sigma | pp ) execution time, O(Max_level + n |Sigma |) memory space and O ( K ) communication rounds. p is the number of processors. Experimental results showed that the parallel algorithm is, respectively, 14.43x and 19.19x faster than the sequential algorithm on 32 and 64 processors.
For latency-sensitive data processing applications in the cloud, concurrent data-parallel tasks need to be scheduled and processed quickly. A data-parallel task usually consists of a set of sub-tasks, generating a set of flows that are collectively referred to as coflows . The state-of-the-art schedulers collect coflow information in the cloud to optimize coflow-level performance. However, most of the coflows, classified as small coflows because they consist of only short flows, have been largely overlooked. This article presents OptaX , a decentralized network scheduling service that collaboratively schedules data-parallel tasks’ small coflows. OptaX adopts a cross-layer, commercial off-the-shelf switch-compatible design that leverages the sendbuffer information in the kernel to adaptively optimize flow scheduling in the network. Specifically, OptaX (i) monitors the system calls (syscalls) in the hosts to obtain their sendbuffer footprints, and (ii) recognizes small coflows and assigns high priorities to their flows. OptaX transfers these flows in a FIFO manner by adjusting TCP’s two attributes: window size and round-trip time. We have implemented OptaX as a Linux kernel module. The evaluation shows that OptaX is at least 2.2× faster than fair sharing and 1.2× faster than only assigning small coflows with the highest priority. We further apply OptaX to improve the small I/O performance of Ursa , a distributed block storage system that provides virtual disks where small I/O is dominant. Ursa with OptaX achieves significant improvement compared to the original Ursa for small I/O latency.
