Large language models (LLMs) require inference systems that can handle both compute- and memory-intensive workloads. GPUs and NPUs (referred to as xPUs) efficiently process compute-intensive layers, while Processing-In-Memory (PIM) architectures are well suited for memory-bound stages. To exploit this complementary relationship, recent LLM inference systems have adopted heterogeneous architectures integrating xPUs with PIM units. However, this integration poses several challenges. Tight execution dependencies between the two devices limit concurrency, as PIM often must wait for data produced by the xPUs. Moreover, batch size and sequence length affect the computational load on PIM and the accelerator differently, leading to execution imbalance across devices and degrading their utilization. To address these challenges, we propose LibraPIM, a novel PIM framework that orchestrates workload rebalancing and concurrent execution between compute accelerators and in-memory compute units, enabling efficient and scalable LLM inference. LibraPIM addresses the above challenges through two key techniques: Dynamic Batch Offloading (DBO), which adaptively redirects portions of the workload to the underutilized device based on runtime profiling, and Dual-Path Execution (DEX), which enables concurrent PIM operations and memory accesses through sub-bank partitioning with minimal hardware overhead. Together, these techniques improve resource utilization across heterogeneous devices, thereby increasing system throughput in LLM inference. Our experimental results demonstrate that LibraPIM achieves $6.2 \times$ average speedup over the baseline PIM-enabled system and delivers a $2.1 \times$ average speedup compared to a state-of-the-art approach.
Sparse-sparse matrix multiplication (SpGEMM) is a well-studied problem on CPUs, GPUs, accelerators (e.g. FPGAs), and distributed systems. The main computational bottleneck in SpGEMM is the reduction process, which involves matching indices to accumulate partial products that map to the same output locations and requires irregular memory accesses. Efficient implementations must use the memory hierarchy effectively so that this reduction is done in fast local (cache) memory as much as possible. This is challenging, especially on GPUs, where the local memory is managed explicitly, as different rows of the result may have vastly different numbers of nonzero elements and vastly different numbers of partial products required to produce the output row which may not fit in the local memory. We demonstrate an SpGEMM implementation on GPUs which perfectly partitions the partial products into equal-size blocks such that each block maps to disjoint output locations. In this way, the block size can be chosen to maximize local memory, all reductions happen in local memory, and each block can be processed independently without communication or data from other blocks. We show how this partitioning can be achieved by solving many instances of a multiway merge partitioning problem. There are several algorithms in the literature for solving this problem. We present the mathematical formulation, missing from all papers providing algorithms for this problem, and show that this is a useful framework for parallelizing it efficiently for GPUs. To our knowledge, this partitioning scheme has never been applied to SpGEMM. We evaluate our SpGEMM implementation, MMSpGEMM, with respect to state-of-the-art implementations including cuSPARSE, TileSpGEMM, spECK, and AC-SpGEMM, achieving speedups of up to $5.3 x, 10.0 x, 1.3 x$, and $1.1 x$, respectively, on a select set of 20 SuiteSparse matrices, and much higher on synthetic matrix multiplications with different left and right matrices. Finally, we discuss improvements and limitations and suggest ways to incorporate the idea into more general SpGEMM routines.
Optimizing convolution operations is critical for enhancing the performance of convolutional neural networks (CNNs). The Winograd convolution algorithm, renowned for its significant reduction in computational complexity, has been widely adopted in convolution acceleration. While the Winograd convolution algorithm has been highly optimized for traditional computing platforms such as SIMD-based CPUs and SIMT-based GPUs, research on alternative architectures better suited for convolution operations remains ongoing. MIMD architectures, characterized by their high parallelism and thread divergence mitigation capabilities, present promising potential; nevertheless, their applicability and performance in convolution operations remain underexplored. Additionally, GPU-based convolution acceleration, despite delivering exceptional performance, faces escalating energy consumption challenges, necessitating a balanced optimization of hardware performance and energy efficiency. To address these, we implement and optimize the Winograd convolution algorithm on a low-power MIMD many-core processor PEZY-SC3s, aiming to investigate its viability for convolution workloads. Experimental evaluations using convolutional layer parameters from widely adopted CNNs demonstrate: 1) 78.37% average bandwidth utilization in data-intensive stages; 2) 98% single-core and 92.52% system-wide computational efficiency in compute-intensive phases. The optimized Winograd algorithm achieves significantly superior computational efficiency compared to cuDNN-based implementation on Nvidia A100 GPU and oneDNN-based implementation on Intel Xeon Silver 4314 CPU, with energy efficiency ratios (GFLOPS/W) of $2.58 \times$ and $27.52 \times$, respectively.
As Deep Learning (DL) training has emerged as a dominant workload on clusters in recent years, there has been significant interest in parallelizing individual models and scheduling them efficiently. However, to date, no prior work has taken an integrated approach to addressing both problems while simultaneously accounting for heterogeneity in the execution environment. This paper introduces, demonstrating that the integrated optimization of parallelization and scheduling is both feasible and highly effective. The main insight of our work is to generate a set of possible target resources for each job and to develop corresponding parallelization strategies for each. Each jobstrategy pair is then treated as a scheduling candidate. Scheduling decisions are made by extending prior Linear Programming (LP)based formulations and incorporating new heuristics to reduce decision overhead.
Graphics Processing Units (GPUs) offer high computational throughput and memory bandwidth, making them promising accelerators for relational OnLine Analytical Processing (OLAP). GPU-accelerated relational OLAP executes the relational operations of a Structured Query Language (SQL) query on a GPU instead of the host Central Processing Unit (CPU). Depending on where input columns and their values reside in, a GPU-accelerated SQL query execution can be classified into two scenarios: 1) in-GPU, in which all input columns fit in the GPU memory, or 2) in-host, in which the input columns reside in the host memory and get transferred to the GPU memory when needed. However, both scenarios incur significant intraGPU and host-to-GPU data movement overheads, respectively. In-GPU executions incur excessive GPU cache misses and thus frequent off-chip GPU memory accesses. In-host executions suffer from the limited host-to-GPU data transfer bandwidth. This paper presents DMO-DB, a Data Movement-Optimized GPU-accelerated relational OLAP engine. Since modern GPUaccelerated relational OLAP decomposes SQL queries into multiple pipelines-sequences of relational operations that can be executed on input columns from the same table, DMO-DB leverages inter-pipeline dependencies to overcome the two data movement bottlenecks. DMO-DB introduces two key ideas: cache-fit bloom filtering and Ahead-of-Time value Discarding (AoTD), which preemptively eliminate unnecessary input values before their movement across the memory hierarchies. For in-GPU execution, GPU L1 data cache-fit filters discard non-contributing values before triggering costly off-chip DRAM accesses. For in-host execution, host CPU last level cache-fit filters strategically prune unnecessary input values, minimizing PCIe transfer overhead. After that, AoTD exploits multiple inter-pipeline dependencies by collecting these cache-fit bloom filters to earlier pipeline execution stages. Our evaluation using NVIDIA RTX A4000 and TITAN RTX GPUs shows that DMO-DB achieves speedups of $\mathbf{1. 5 3 x}$ over in-GPU Crystal-Opt and 6.10x over in-host HeavyDB.
Contemporary high performance computing, cloud, and embedded systems are equipped with various combinations of heterogeneous processors. This has led to the development of portable abstractions to target these heterogeneous systems. However, these abstractions fail to deliver performance portability as they cannot easily modify key parameters such as the amount of concurrency, the representation of the computation graph, the kernel implementation, and data transfers for varying problem sizes and hardware architectures. To address this wide parameter space, this work presents IRISX, a framework that dynamically finds the performance portable solution based on application characteristics and underlying hardware. By changing the representation of the computation at the kernel level, task level, and graph level, IRISX finds the appropriate set of heterogeneous processors that provides the best performance while ensuring portability to different heterogeneous systems.
This work introduces AXIOS, a novel spatial architecture for accelerating hash-based post-quantum cryptography (PQC) primitives. AXIOS demonstrates that structural regularities in hash-based algorithms can be efficiently mapped to spatial accelerators that support FPGA-based programming for granular control and coarse-grained reconfigurable arrays (CGRA) for repeated tasks. AXIOS selects the key generation task of the eXtended Markle Signature Scheme (XMSS), which embodies critical implementation challenges in modern hash-based (PQC) algorithms. The AXIOS implementation on AMD’s VCK190 platform demonstrates an $8.54 \times$ improvement in runtime and a $71.65 \times$ improvement in energy efficiency compared to a benchmark implementation on Intel’s Core $\mathbf{i 9 - 1 4 9 0 0 K}$. AXIOS also breaks the current record of XMSS acceleration in terms of execution time on an embedded SoC or an FPGA platform. To our knowledge, this is the first efficient hardware implementation of compute-intensive hash-based PQC schemes in an embedded spatial architecture. Albeit complex, this FPGA+CGRA-based design is a promising step to support compute-intensive PQC applications at the edge. This work’s code and experimental artifacts are publicly available at https://github.com/SPIRE-GMU/AXIOS.
The Stable Marriage Problem (SMP) is a combinatorial optimization problem aimed at creating stable pairings between two groups, traditionally referred to as men and women. SMP has been widely applied across various domains, including healthcare, education, and cloud computing, to optimize resource allocation, matching, and utilization. The classical approach to solving SMP is based on the Gale-Shapley algorithm, which constructs stable pairings sequentially. However, this algorithm is both time-consuming and data-intensive, resulting in significant slowdowns even for a moderate number of participants. Despite various attempts to parallelize the Gale-Shapley algorithm over the years, progress has been consistently impeded by three major bottlenecks: (1) frequent and expensive data movement, (2) high synchronization overhead, and (3) workload-dependent and irregular data accessing and parallel processing patterns. To resolve the above-mentioned bottlenecks, in this paper, we introduce Bamboo-SMP, a highly efficient parallel SMP algorithm, and its implementation in a hybrid environment of GPUs and CPUs. We have made three key development efforts to achieve high performance for Bamboo-SMP. First, Bamboo-SMP effectively exploits the data accessing locality with a lightweight data structure to maximize the “shared residence space”. Second, Bamboo-SMP employs an advanced hardware atomic operation to decrease execution latency with “low contention”. Third, Bamboo-SMP is implemented in a hybrid environment of GPU and CPU, leveraging the high bandwidth of the GPU for massive parallel operations and the low latency of CPU for fast sequential tasks. By fostering “mutual complementarity” between CPU and GPU, Bamboo-SMP attains superior performance, consistently exceeding the best existing methods by 6.69 x to 21.44 x across a wide range of workloads. Moreover, Bamboo-SMP demonstrates excellent scalability, efficiently solving large-scale SMP instances while achieving sustained speedups of $5.6 \times$ to $13.8 \times$ on 4 GPUs. To the best of our knowledge, Bamboo-SMP is the fastest and most scalable solution for SMP data processing.
Recently, we have observed a steady growth in specialized hardware accelerators. These accelerators are typically programmed in high-level domain-specific languages (DSLs), enabling compilers to generate efficient code for rapidly evolving heterogeneous hardware. However, rewriting existing code to exploit DSL compiler performance is an onerous programmer task. This has led to recent interest in automatically translating or lifting code to DSLs. Current lifting techniques use language models or program synthesis to translate code. Although language models have proved remarkably successful in related translation tasks, they are prone to hallucinations. Program synthesis approaches are accurate but do not scale to complex tensor DSLs. This paper presents a novel approach, Guess, Measure & Edit; that exploits both language models and compiler technology to lift existing code to high-level DSLs. Given a source program, it uses a language model to guess an initial equivalent target program. It then compiles or lowers the guess and the original program, and measures the low-level distance between them using program similarity metrics. It iteratively uses these low-level metrics to guide high-level edits to the guess until it is correct. To validate this approach, we develop KONRUL which correctly lifts existing tensor algebra C code to einsum notation, the basis of tensor contraction DSLs. Our evaluation shows that KONRUL is fast and accurate, lifting 98% of an extensive benchmark suite and significantly outperforming 4 state-of-theart lifting schemes. KONRUL is scalable and is the only approach to correctly lift higher-dimensional tensor contraction code. Our lifted programs result in geomean speedups of $4.07 \times$ and $38.30 \times$ when ported to a multi-core CPU and GPU respectively.
We introduce a two-level scheme to generate GPUaware MPI/NCCL code for distributed tensor computations. Our generator takes the specification of a linearized Directed Acyclic Graph (DAG) of tensor operators and produces a global mapping solution that considers MPI communication (inter and intranode) and the local computation. The core of our generator is a new bit-vector representation that compactly models mappings as well as communication directions along the grid. We incorporate the 2-level mapping decisions into a non-linear formulation which is optimized in an iterative fashion with the Z3 SMT solver. The new mapper supports both NVIDIA NCCL, MVAPICH-gdr, allowing for better portability. We demonstrate the efficiency of our mapping generator on a set of matrix- and tensor- DAGs, on two multi-GPU clusters with NVLink or PCIe intra-node interconnect, and compare against the COSMA library and CTF framework, achieving speedups ranging from 2.6× (over COSMA) to 18× (over CTF).
The Compute Express Link (CXL) technology facilitates the extension of CPU memory through byte-addressable SerDes links and cascaded switches, creating complex heterogeneous memory systems where CPU access to various endpoints differs in latency and bandwidth. Effective tiered memory management is essential for optimizing system performance in such systems. However, designing an effective memory tiering system for CXL-extended heterogeneous memory faces challenges: 1) Existing evaluation methods, such as NUMA-based emulation and full-system simulations like GEM5, are limited in assessing hardware-based tiered memory management solutions and handling real-world workloads at scale. 2) Previous memory tiering systems struggle to simultaneously achieve high resolution, low overhead, and high flexibility and compatibility. In this study, we first introduce HeteroBox, a configurable emulation platform that leverages real CXL-enabled FPGAs to emulate the performance of various CXL memory architectures. HeteroBox allows one to configure a memory space with multiple regions, each exhibiting distinct CPU-access latency and bandwidth. HeteroBox helps assess the performance of both software-managed and hardware-managed memory tiering systems with high efficiency and fidelity. Based on HeteroBox, we further propose HeteroMem, a hardware-managed memory tiering system that operates on the device side. HeteroMem creates an abstraction layer between the CPU and device memory, effectively monitoring data usage and migrating data to faster memory tiers, thus hiding device-side heterogeneity from the CPU. Evaluations with real-world applications show that HeteroMem delivers high performance while keeping heterogeneous memory management fully transparent to the CPU, achieving a $\mathbf{5. 1}$ % to $\mathbf{1 6. 2}$ % performance improvement over existing memory tiering solutions.
The increasing prevalence and growing size of data in modern applications have led to high costs for computation in traditional processor-centric computing systems [1–20]. To mitigate these costs, the processing-in-memory (PIM) $[1,2,5,6$, 8,9,20-27] paradigm moves computation closer to where the data resides, reducing the need to move data between memory and the processor. Even though the concept of PIM was first proposed in the 1960s [24, 28], and various PIM architectures have been proposed since then [$10,17,18,20,29-65$], realworld PIM systems have only recently been manufactured [6670], among which the UPMEM PIM system [66, 67, 71] is the first PIM architecture to become commercially available. A general-purpose PIM system is often composed of regular DRAM (as its main memory) and specialized PIM DRAM DIMMs. A PIM module is a standard DDRx DIMM (module) with multiple PIM chips. Inside each PIM chip, there are multiple (e.g., 8) general-purpose in-order PIM cores, which have exclusive access to a DRAM bank and SRAM-based instruction/scratchpad memories.
LLM deployment on end-user devices has attracted significant interest from tech giants and research institutions due to privacy benefits and the elimination of network roundtrips. Reducing latency is crucial for improving user experience. Enduser devices, such as desktop and mobile processors, often integrate CPU and GPU on a single die, making tensor parallelism a promising approach to distribute workloads and reduce inference latency. However, the predefined tensor parallelism in traditional practices cannot guarantee optimization due to heterogeneity and resource contention in the CPU-GPU integrated end-user devices. In this paper, we propose Doppeladler, a practical framework designed to facilitate parallel inference of LLM on end-user devices. Doppeladler adaptively optimizes tensor parallelism based on real-time conditions and device status, enhancing resource utilization by refining workload partitioning and scheduling with a heuristic-based approach. Additionally, it lowers the costs of CPU-GPU hybrid execution by managing device resource, trigger workload rebalance during runtime to mitigate the penalty of contention, and minimizing communication overhead through zero-copy techniques and alternate access pattern. Experiments demonstrate that Doppeladler outperforms existing methods by 1.2 to 2.5 times.
Finite Automata (FA) processing is a core computation in various real-world applications. Over the past decades, extensive efforts have been dedicated to accelerating FA processing on modern parallel platforms, particularly GPUs, due to their high memory bandwidth and massive hardware parallelism. As Non-deterministic Finite Automata (NFA)-based applications have strong and growing demands for real-time data analytics nowadays, reducing latency in automata processing has become a critical priority. However, existing approaches face significant challenges when limited parallelism is exposed in NFA computations. In this work, we explore opportunities of introducing fine-grained parallelism from various sources and addressing the limitations of fast NFA processing. Specifically, by analyzing different NFA parallelization schemes, we identify the major performance issue caused by insufficient state-level parallelism in conventional designs. To overcome the bottleneck, this work introduces speculative parallelization tailored for GPU-based NFA processing, thus effectively exploiting fine-grained parallelism across multilevels, with a particular focus on input-chunk-level parallelism. To realize speculative parallelization in practice, we develop $A N G$, a latency-oriented NFA processing framework that overcomes key implementation challenges on GPUs. We evaluate the efficiency of ANG on a set of representative NFAs with diverse properties. Experimental results demonstrate that ANG achieves significant performance improvement compared to state-of-theart techniques, with reaching $11.74 \times$ speedup on average (and up to $49.88 \times$ in extreme cases).
Modern high-performance computing (HPC) platforms rely on hybrid combinations of shared-memory and distributed-memory programming models to achieve scalable parallelism. Despite their success, these hybrid approaches remain challenging for many programmers trained primarily in shared-memory models such as OpenMP. For decades, researchers have sought ways to automatically generate distributedmemory parallelism from shared-memory programs to bridge this productivity gap. The Partitioned Global Address Space (PGAS) approach, exemplified by UPC and OpenSHMEM, represents one such effort, but its reliance on Bulk Synchronous Parallel (BSP) execution often results in poor performance for irregular algorithms, such as those found in graph applications. This paper presents a compiler-driven approach that automatically generates high-performance distributed-memory code from a simple PGAS-style extension to OpenMP, referred to as PGAS-OpenMP. The generated code employs an actor-based distributed-memory execution model, which has been shown to significantly outperform BSP implementations for irregular workloads. While BSP-style PGAS code can always be generated in a straightforward manner from PGAS-OpenMP, actor-based translation requires careful analysis to ensure correctness. Our experimental results demonstrate that the proposed compiler safely converts a wide range of PGAS-OpenMP graph applications into asynchronous actor-based code, yielding substantial performance improvements. Unlike prior work on translating shared-memory programs to distributed-memory systems, this is the first approach to automatically target actor-based execution for programs with irregular parallelism. The generated code supports fine-grained asynchronous one-sided messaging with automatic message aggregation, enabling efficient communication and computation overlap. This research addresses a long-standing challenge in HPC combining programmer productivity with high performance for irregular distributed-memory applications and provides a promising pathway toward unifying shared-memory programmability with actor-based distributed-memory execution for current and future HPC systems.
Structured-mesh-based stencil computations are a common motif in many numerical algorithms, such as for solving PDEs. Recent work has shown promising runtime performance and energy efficiency when mapping these applications to FPGAs. However, this requires significant manual effort with hardwarespecific optimizations and customization. We present a new codegeneration framework that automates the mapping of stencil applications to FPGAs. From a domain-specific declaration, the framework applies radical, ad-hoc, and dynamic optimizations, including a novel window-buffer chaining scheme, and an on-chip loopback pipeline approach that minimizes memory transaction overhead. We show that the generated code is able to match or exceed the performance of hand-tuned state-of-the-art designs. A range of stencil solvers benchmarked, including non-trivial multi-stencil solvers, on two FPGAs, demonstrating performanceportability. Comparisons to best-in-class solvers on an Nvidia H100 GPU indicate competitive performance with 2-19× energy savings.
Analyzing and understanding the memory access behaviors of applications are essential when optimizing computing systems and applications. One prominent example is the cache miss curve estimation, i.e., detecting the cache miss ratio or frequency as a function of the capacity using a memory access sequence. Historically, cache miss curves have been derived from their corresponding stack distance distributions that are obtained by keeping track of the depth on the LRU stack. As this procedure requires significant computational complexity, a variety of approximation techniques have been proposed ever since it was originally proposed by Mattson et al. in 70s. We, however, claim that stack distances are not necessarily required to derive a cache miss curve.This paper proposes an alternative, efficient approach: instead of relying on the stack distances, we approximate a cache miss curve from the cardinality of accesses - total access count as a function of unique access count. By doing so, we can apply wellestablished efficient probabilistic data structures (e.g., Log-Log counting and its variants) widely used in various domains. In our approach, we model LRU stack behavior as a series of Bernoulli trials and derive a macroscopical relationship between a cache miss curve and its corresponding cardinality curve. Driven by this relationship, we offer our miss curve estimation mechanism using cardinality detection data structures and a curve fitting approach. We further offer several advanced techniques by taking advantage of the cardinality domain: (1) a compensation mechanism for sampled memory traces and (2) an algorithm to synthesize a miss curve of multiple access sequences by following the additivity principle in the cardinality domain. We comprehensively validate our approach using SPEC CPU 2017 benchmark suite under a variety of scenarios.
Fully Homomorphic Encryption (FHE) enables computation directly on encrypted data, and multiple hardware accelerators have emerged. Unfortunately, each accelerator is evaluated using a rigid and proprietary simulator. This makes it challenging to compare accelerators or evaluate the performance impact of individual ideas. Perhaps more importantly, it has proven to be challenging to transfer ideas across accelerators. Each new idea in FHE typically requires changes across the stack, from the security protocol (e.g., KeySwitching), to the memory hierarchy (e.g., on-the-fly key reuse), and individual functional units (e.g., FFT-based polynomial multipliers). Currently, there is no tool for exploring FHE accelerators. We introduce FLASH, an extensible, modular, and opensource simulator for FHE accelerators. We pioneer a new methodology for constructing architecture simulators-compilerlike dialects (or intermediate representations). FLASH includes three distinct yet interoperable dialects for writing imperative code on encrypted types (Security), expressing spatial and temporal data movement of polynomials (PolyDataflow), and creating custom functional units (Modular arithmetic). A transpiler seamlessly ties these dialects, and rapidly high-level-synthesizes (HLS) them to accelerator-specific performance model. Users can implement algorithm-hardware co-design ideas, as transformations of kernels written in our dialects, and do not require time-consuming buggy simulator rewrites. We demonstrate the versatility of FLASH using multiple case studies: i) we conduct the first pubic head-to-head comparison of multiple FHE accelerators. ii) we backport ideas from multiple papers and quantify their impact (e.g., KeySwitching). We can construct an FHE accelerator model from scratch with $\simeq \mathbf{5 0 0}$ lines-of-code (vs. 3000+ when working with existing FHE library and simulator). iii) Finally, we show that we can implement new ideas with modular changes ($\lt50$ lines of code). The dialects helps us affect the changes in a targeted and concise manner.
Heterogeneous computing has become increasingly prevalent, driven by the huge computational demands of highperformance computing (HPC) and artificial intelligence (AI) workloads. Yet, evaluating these workloads on modern systems poses significant challenges. Existing tools for the instrumentation and analysis of CPU and GPU applications run separately, introducing timing differences and limiting the ability to capture their runtime interactions. To address this problem, we introduce XPU-Pin, a framework that enables simultaneous CPU and GPU binary instrumentation in a single execution. XPU-Pin integrates the CPU instrumentation framework Pin with GPU instrumentation frameworks such as NVBit (for NVIDIA GPUs) and GTPin (for Intel GPUs). This approach allows for the holistic analysis of heterogeneous workloads irrespective of the platform it executes. Leveraging the co-analysis capabilities of XPU-Pin, we present a novel methodology called XPU-Point to select simulatoragnostic representative samples of heterogeneous CPU-GPU workloads. The XPU-Point methodology employs tools developed with the XPU-Pin framework to: (a) capture the execution signature of heterogeneous programs and (b) evaluate the accuracy of selected samples on silicon, which were not possible before. We evaluate XPU-Point on diverse hardware platforms (x86 CPU with Intel/NVIDIA GPUs) using workloads such as SPECaccel 2023, SPEChpc 2021, GROMACS, AutoDock, and PyTorch. We demonstrate that XPU-Point predicts overall application performance with sampling errors typically less than $\mathbf{5} \boldsymbol{\%}$ as measured on native hardware.
Retrieval-Augmented Generation (RAG) systems typically rely on dense floating-point embeddings to retrieve relevant documents, but this approach incurs significant memory and compute costs at scale. We propose a Hyperdimensional Computing (HDC) framework that projects transformer token embeddings into high-dimensional binary hypervectors, which are aggregated into compact document representations. To support sublinear search, we introduce HD-NSW, a graph-based index inspired by navigable small-world networks. HD-NSW clusters similar hypervectors into bundled centroids and connects them with sparse Hammingdistance edges, enabling efficient, beam-guided traversal entirely in the binary domain. Across 15 BEIR benchmarks and synthetic Gaussian mixture corpora, HD-NSW achieves over 99% of dense retrieval quality, reduces memory usage by $8 \times$, and supports over 860 queries per second at 10 million documents while maintaining over 80% throughput at 40 million documents. At five million documents, HD-NSW achieves $7.68 \times$ higher throughput compared to state of the art approximate nearest neighbor methods. Beyond this point, competing baselines encounter memory exhaustion, while HD-NSW continues scaling and maintains high throughput at larger corpus sizes.
