
While NAND flash memory has continuously increased its storage density over decades, this progress has exacerbated the read-disturbance problem. In this work, we identify two fundamental limitations of existing read-disturbance management techniques that trigger read reclaim (RR) at the block granularity: (i) they overlook the heterogeneous reliability impact of read disturbance across individual word-lines (WLs), leading to unnecessary RR in many cases; and (ii) they address read disturbance only after disturbance-induced errors have already accumulated, which forces substantial RR-induced copy overheads in read-disturbanceprone modern NAND flash memory. To address these limitations, we propose STRAW (STRess-Aware Wordline-based read-disturbance management), a new technique that minimizes RR overheads through two key ideas: (i) stress-aware WL-based read reclaim, which monitors the accumulated read-disturbance effect on each WL and reclaims only heavily disturbed WLs, and (ii) stress-reduced read, which mitigates disturbance on valid WLs during each read operation by scaling pass-through voltages based on WL validity. Our experimental results using a modern SSD emulator show that STRAW reduces RR-induced page-copy overhead by 88.6% on average compared with the state-of-the-art technique.
As GPUs and other accelerators become increasingly popular, optimizing I/O operations between on-chip and off-chip memory is increasingly critical. I/O analysis, however, is complex, requiring a deep understanding of application dataflow and memory hierarchy. Developing a practical I/O analysis methodology remains a timely challenge. Self-attention is employed extensively in transformer models, but its quadratic memory complexity poses significant challenges to modern memory systems. In this study, we explore how to use I/O analysis to develop optimal solutions for accelerating exact long-sequence self-attention. We first introduce a novel I/O analysis for tall-and-skinny matrix-matrix multiplication, which captures the dominant data movement behavior of long-sequence self-attention. Guided by systematic I/O analysis, we develop AttenIO, an I/O-driven accelerator for exact long-sequence self-attention with three key optimizations: (1) an analytically derived I/O-optimal tiling and scheduling to minimize I/O operations, (2) fine-grained three-level communication-computation overlapping to hide I/O stalls, and (3) parallel execution patterns for efficient softmax. Our evaluation shows that AttenIO achieves a 1.6x8.8x speedup over the state-of-the-art solutions. Although AttenIO is designed for self-attention, it also highlights the broader potential of I/O analysis as a principled foundation for guiding high-performance I/O optimizations.
Data center applications' large instruction footprints cause frequent front-end stalls by overwhelming on-chip micro-architectural structures such as instruction cache (I-cache), instruction translation look-aside buffer (iTLB), and branch target buffer (BTB). To reduce pressure on these structures, data center providers leverage profile-guided optimizations by reordering binary layout along a relatively small number of hot code paths. Such reordering provides the highest benefit if profile collection and optimization happen on the same version of the binary. In practice, companies have to optimize and deploy a fresh version of the binary with a profile from a previous version, making a large fraction of the profile stale. In this paper, we propose Wax(1), a novel technique to optimize data center applications with stale profiles. Wax's key insight is to leverage the debug and source code information while optimizing fresh binaries with stale profiles. We evaluate Wax for 5 data center applications to show that Wax provides significant (5.76%-26.46%) performance speedups. Wax achieves 1.20%-7.86% greater speedups than the state of the art, obtaining 65%-93% of fresh profiles' benefits.
Fully homomorphic encryption (FHE) enables arbitrary computation over encrypted data (ciphertext) without compromising confidentiality. Within this family, TFHE features versatile bootstrapping mechanisms that is attractive for security-critical applications. However, its prohibitive computational cost severely limits practical deployment. While hardware acceleration is promising, mere compute scaling fails to overcome the inherent barriers. In particular, the combination of limited algorithmic parallelism and inadequate understanding of hardware behaviors prevents full exploitation of the available performance headroom. In this paper, we present Maverick, a GPU-based solution that rethinks TFHE bootstrapping acceleration through joint algorithm-hardware co-design. At the algorithmic level, we reconstruct blind rotation by shifting test vectors from pre-loading to post-injection. This reformulation turns the original length-.. single chain into a multi-chain structure with root n sub-chains of depth root n, raising external-product (EP) parallelism from 1 to the theoretical bound of root n. At the hardware level, we introduce a partial-domain transformation strategy that terminates the NTT early in a stage-aware manner and selectively shifts computation to downstream operators, redefining operator boundaries to balance the workload across EP operators. Experimental results show that Maverick delivers state-of-the-art GPU acceleration. It outperforms the best existing CPU and GPU baselines by 331.2x and 3.4x, respectively, on programmable bootstrapping, and achieves up to 108.5x speedup over CPU-based circuit bootstrapping. Together, these results highlight the substantial potential of algorithm-hardware co-design for advancing FHE performance on general-purpose platforms.
Low-latency, single-request decoding of large language models is critical for interactive systems with tight SLA demands. Prior work reduces latency through speculative decoding (combining a small draft model with a larger target model), but the draft model remains on the critical path, and communication overhead limits scaling across GPUs due to the small batch size associated with single-request decoding. To address these limitations, this paper introduces SwiftSpec: a system architecture that disaggregates draft and target models across homogeneous GPUs within a single node and utilizes NCCL-low-latency primitives directly to improve the performance of core GEMM and attention kernels. Our implementation includes 3k lines of custom CUDA for fused kernels and an evolving tree cache for KV-cache consistency and maximized reuse between draft and target models. On a single 8xH800 GPU node, SwiftSpec achieves 347 tokens/s for Llama-3-70B-1.3x faster than NVIDIA's own benchmarks on a higher-performance 8xH200 setup-and averages 1.75x faster decoding than state-of-the-art speculative decoding across five model families and six datasets. Specifically, we find that for Llama-3-70B SwiftSpec is significantly faster across all 480 tested queries, showing 1.7x speedup over the best open-source baseline for 95th percentile requests.