Lease caches make eviction prescriptive by assigning a compiletime or profile-derived lease to each memory reference, enabling program-guided cache management. This paper presents a reproducible hardware-software evaluation of lease-based cache programming: we implement a lease-cache prototype (CLAM and SHEL) on a single-core RISC-V emulated on an Altera Cyclone-V FPGA and compare its measured behavior to a PLRU baseline and to Belady's OPT computed by an offline Rust simulator. We collect sampled reuse statistics and full traces from PolyBench/C kernels to produce per-reference leases and optimal miss counts. Lease programming closes most of the gap to OPT (average normalized score 67/100), with multi-scope (SHEL) assignments giving higher and more stable gains than single-scope (CLAM); triangular iteration spaces remain the hardest class to optimize. We also show lease-cache rationing is robust to modest under-allocation. The results motivate practical use of lease programming in resource-constrained, fixed-size caches and point to opportunities for runtime adaptation to handle highly irregular access patterns.
Historically, software engineers and hardware engineers represent disparate groups that don't talk to each other. Because the two don't interact much, neither knows much of the other's discipline, and therefore those they teach learn little of the other's discipline. This is problematic because computer design (hardware) and computer security (software) are intimately dependent on each other, and when system designers fail to understand both topics, catastrophe results. For example, critical vulnerabilities like buffer overflows and Spectre/Meltdown arose because designers inadvertently bypassed protections that could have ensured robust security. Moreover, teaching security and hardware as separate disciplines obscures the root causes of vulnerabilities, leading to solutions that fail to tackle the fundamental flaws. This paper shows howan understanding of both hardware and security will help one to recognize the dependences and dangerous gaps that exist between the two. Hopefully such awareness can help prevent the future undermining of system security via unintended design consequences.
Machine learning (ML) workloads (e.g., deep neural networks (DNNs), convolutional neural networks (CNNs)) have grown significantly in size and complexity. With such growth in ML workloads, memory performance has emerged as a critical bottleneck in specialized hardware accelerators (e.g., AI accelerators) for ML workloads. The AI accelerators feature tightly-coupled on-chip SRAM and compute units in systolic array architectures that demand high memory reuse efficiency to maintain throughput. In this paper, we present a comprehensive analytical memory modeling framework that leverages reuse distance analysis to predict SRAM-level behavior for ML workloads on AI accelerators. Our model extends traditional reuse distance analysis by incorporating dataflow semantics, spatial and temporal reuse dynamics, and architectural buffer constraints. Using our model, we measure and report key memory metrics such as SRAM hit rate, miss rate, bandwidth utilization, and latency. The metrics are also derived from trace-level analysis generated by accelerators like ScaleSim. We evaluate our model across input-stationary, weight-stationary, and output-stationary mappings across a broad range of DNN and CNN benchmarks on AI accelerator, showing how reuse behavior varies by dataflow architecture and applications. Our results offer architectural insights that inform efficient buffer sizing and dataflow configuration to maximize on-chip memory efficiency and reduce off-chip traffic.
Efficient memory access patterns play a crucial role in determining the overall performance of applications by exploiting temporal and spatial locality, thus maximizing cache locality. The Reuse Distance Histogram (RDH) is a widely used metric to quantify temporal locality, measuring the distance between consecutive accesses to the same memory location. Traditionally, calculating RDH requires program execution and memory trace collection to obtain dynamic memory access behavior. This trace collection is often time-consuming, resource-intensive, and unsuitable for early-stage optimization or large-scale applications. Static prediction, on the other hand, offers a significant speedup in estimating RDH and cache hit rates. However, these approaches often lack accuracy, as predictions are made without executing the program or observing the complete memory access pattern. Specifically, when arrays are used inside nested loops, predicting the access patterns of array references without observing their exact execution sequence presents a significant challenge. This paper presents a novel static analysis framework for predicting the reuse profiles of array references in programs with nested loop structures, independent of runtime information. By analyzing loop bounds, access patterns in smaller problem sizes, and predictive equations, our method predicts array access patterns and estimates reuse distances and cache hit rates at compile time. This paper extends our previous study by incorporating a deeper analysis and improving prediction by addressing previously unhandled reuse patterns. We evaluated our technique against a widely accepted traditional trace-driven profiling tool, Parallel Reuse Distance Analysis (PARDA). The results demonstrate that our static predictor achieves comparable accuracy while offering orders-of-magnitude improvement in analysis speed, excluding setup time. This work presents a practical alternative to dynamic reuse profiling for array applications with multiple loops, paving the way for integration into compilers and static performance modeling tools.
Hardware exploits such as Spectre and Meltdown underscore the increasing need to validate hardware security properties. To account for known exploits we suggest Hybrid logic, an extension of Modal logic, for its concise description of security problems. We further suggest that compiling known exploits into a database would be a helpful tool for testing designs against security properties. We demonstrate a hybrid logic description for Spectre and describe its benefits. We reflect further on the unique challenges of sharing security information about hardware vulnerabilities. We conclude that careful thought is required to achieve responsible disclosure processes for hardware vulnerabilities.
FPGA-based emulation provides a reconfigurable and efficient environment for evaluating SoC memory systems prior to full silicon deployment. Increasingly used to accelerate HPC and AI workloads, FPGAs enable hardware-level exploration of performance-critical subsystems such as memory, making them ideal platforms for proto-typing accelerators in today's heterogeneous computing landscape. Due to modern HPC and AI workloads' increasing bandwidth and energy requirements, understanding and optimizing off-chip memory behavior, especially with High Bandwidth Memory (HBM), becomes critical. In this position paper, we present the ongoing development of Memory Sandbox 2.0, a modular, open-source framework for analyzing memory behavior across both performance (throughput, latency) and telemetry (temperature, power). Memory Sandbox 2.0 enhances the original tool with increased configurability, modular extensions for emerging technologies such as HBM2e, flexible support for varied access patterns, and integrated telemetry monitoring to guide energy-aware system design. We aim to support design space exploration of memory-access behavior and its architectural implications across platforms. While full experimental validation is still underway, we share early insights into challenges faced while profiling HBM2 and HBM2e on cutting-edge AMD Xilinx FPGAs such as the Versal V80 and Alveo U280. We also describe key architectural changes in recent HBM systems and how our tool models and exposes their behavior. Memory Sandbox 2.0 helps identify architectural bottlenecks and trade-offs early in the design cycle by enabling reproducible, insightful evaluation of memory-access patterns and telemetry data. This contributes to more effective and energy-efficient FPGA-based accelerator development, paving the way for more performant and energy-efficient HPC and AI solutions.
Side-channels are unintended pathways through which computer systems leak information, originating from physical or microarchitectural behaviors that produce observable phenomena correlated with internal operations. Despite decades of research, the field suffers from inconsistent terminology and classification. This work proposes a structured framework to define and categorize side-channel phenomena based on leakage source and functional activity: Side-Channel Attack (SCA), Side-Channel Analysis (SCAN), and Side-Channel Defense (SCD). The utility of this taxonomy is demon-strated through a case study of microarchitectural SCAN using Hardware Performance Counters (HPCs), which capture low-level CPU events. HPCs are shown to enable passive, low-overhead monitoring for early detection of ransomware and other malware. Ongoing work explores cross-platform applicability, improved event selection, and the integration of machine learning for real-time anomaly detection, advancing the role of HPC-based side-channel analysis in practical security applications.
This paper examines the security and forensic implications of Solid State Drives (SSDs), emphasizing howtheir architectural complexity enables both covert channels and active attacks. We survey methods for covert storage-such as data remanence in over-provisioned space, FTL manipulation, and analog-state exploitation-as well as timing channels that exploit performance modulation. We also analyze active threats, including firmware-based device impersonation and DMA-level exploits over NVMe. However, beyond these risks, SSDs also introduce underappreciated or unintended security and forensic advantages. Features such as built-in hardware encryption, rapid secure erase, the absence of residual magnetism, and inherent resistance to certain physical side-channel attacks offer meaningful benefits in defensive contexts. These advantages, while not always designed with security as the primary goal, can be leveraged to improve data confidentiality, integrity, and forensic reliability. As SSD technologies continue to evolve-and with them, emerging paradigms like ReRAM and MRAM-the answer to the question "Is SSD a friend or a foe?" lies not in the medium itself, but in how its capabilities are harnessed-or weaponized-by both attackers and defenders.
This paper introduces TGN-PNM, a Processing-in-Memory (PIM)-based accelerator designed specifically for Temporal Graph Neural Networks (TGNNs). TGNNs are gaining increasing attention due to their ability to capture complex relationships and temporal dynamics in various domains. However, designing accelerators for TGNN workloads poses several challenges, including the lack of a standard model architecture, the absence of distinct execution phases, and the difficulty in maintaining workload balance in evolving graphs. Existing accelerators for static GNNs are not easily extendable to TGNNs. In this work, we propose TGN-PNM, which leverages the concept of vault-level parallelism by placing a Vault Processing Unit (VPU) at each vault in a 3D-stacked memory. The VPU consists of a SIMD unit for memory-intensive operations and a systolic array for compute-intensive operations. By placing compute units at the logic layer, our design achieves near-linear performance improvement with increasing memory stacks and exposes higher internal memory bandwidth. We address the challenges of TGNN workloads by introducing a feature-dimension partitioning scheme that minimizes inter-vault communication and improves workload balance. Our architecture allows compute units in all vaults to work in lock-step, resulting in efficient execution. We evaluate TGN-PNM using various TGNN models, batch sizes, and datasets, demonstrating its effectiveness in handling both memory-bound and compute-bound kernels. Our proposed accelerator offers significant performance improvements over existing approaches and provides flexibility to accommodate future TGNN model variations.
The escalating cost of DRAM and the typically high compressibility of memory content make main memory compression highly desirable. However, its practical deployment has been hindered by significant challenges, including its adverse impact on performance and, more critically, the substantial integration challenges it poses to computing infrastructure. The emerging Compute Express Link (CXL) ecosystem provides a unique opportunity to implement main memory compression with minimal integration overhead, shifting the primary adoption barrier towards performance impact. This paper tackles this challenge by introducing three simple yet effective design techniques to enhance the design of compression-capable CXL memory controllers. The first two techniques improve the trade-off between compression ratio and speed performance by dynamically adjusting compression configurations in adaptation to runtime data characteristics. The third technique mitigates compression-induced speed performance degradation by decoupling the in-memory placement of compressed data blocks from their associated error correction code (ECC) redundancy. To evaluate these techniques, we performed RTL-level design and synthesis to estimate silicon cost overhead and developed a simulation platform to capture the trade-offs between compression ratio and speed performance. The results demonstrate that the proposed techniques effectively improve compression ratio vs. performance trade-offs with negligible silicon cost overhead.
Memory hierarchy often represents a significant performance bottleneck in modern computing systems. A promising direction to mitigate this bottleneck is through HW/SW coordination at the system level. However, many existing solutions require changes to legacy programming paradigms, such as ISA extensions, and often provide specialized optimizations limited to specific modules or policies within the memory hierarchy. In this work, we introduce InterStellar, a HW/SW co-design methodology that overcomes these limitations. InterStellar enables the design of a stream-aware memory controller that dynamically adapts its scheduling and memory management policies while proactively batching future stream accesses from off-chip memory. The design is optimized not only for performance, but also for energy efficiency and bandwidth utilization. On systems with eight RISC-V cores, InterStellar achieves significant end-to-end speedup compared to a commercial off-the-shelf (COTS) memory controller: up to 2.72x for PolyBench, 1.84x for HPCG, 1.24x for Rodinia, 1.47x for Parboil, and 1.29x for the Phoenix suite.
Computational complexity characterizes a program's resource requirements, most commonly time and space. Locality complexity characterizes the asymptotic behavior of cache memory as program inputs and cache sizes vary. This paper presents two properties of locality complexity. First, Monotonicity: complexity is non-decreasing with input size, a property proven for fully associative LRU and working-set caches. Second, Proportionality: with optimal cache management, performance improves at least linearly with cache size. This property is universal, holding for all programs and inputs.
Memory disaggregation in data centers has been approaching practicality, owing to the maturity of interconnect standards like Compute Express Link (CXL) [3]. CXL presents a hardware centric approach for multiple compute nodes to pool memory capacities from a shared Fabric Attached Memory (FAM) node, on a per need basis. Using FAM for memory provisioning can potentially mitigate resource underutilization and yield in cost savings, but can cost the application it's performance due to relatively longer access latency. Modern processors attempt to hide memory access latency by employing sophisticated cache prefetchers. While resourceful, current cache prefetching techniques can be further optimized, in light of the long access latency of CXL FAM. To that end, we consider multi-level cache prefetcher that adds additional layer of prefetching at Last Level Cache (LLC). Our multi-level prefetching scheme increases the fraction of requests that hit in LLC, potentially decreasing the sensitivity of workload to FAMlatency. We implemented our multi-level cache prefetcher using SST simulation components [29], and evaluated it with workloads from standard benchmarks suites in single and multi-node system configuration. Our evaluation reveals that, comparing to using only per-core prefetcher, multi-level prefetcher resulted in performance improvement of 2-7%, with LLC hit fraction increasing by 13%.
The increasing performance gap between the bandwidth requirements of modern Systems on Chip (SoCs) and the I/O data rate delivered by Dynamic Random Access Memory (DRAM) limits the performance of today's data-intensive applications. A promising direction to mitigate this bottleneck lies in application-specific DRAM address mappings, which exploit the knowledge of an application's memory access behavior to increase the throughput, reduce energy consumption and lower access latency. In this paper, we propose a framework that generalizes the state of the art, by synthesizing address mappings based on Binary Integer Matrices (BIMs) over GF(2) (i.e. the finite field with two elements), constructed from simple logic operations (XOR and AND) in order to address two key challenges simultaneously: minimizing row misses and constraining hardware complexity. We show that the size of the design space can be significantly reduced by reformulating the row-miss minimization problem for XOR-based mappings as a subspace selection problem over GF(2)(n) that is determined by the bit flip information of the whole address sequence considered. Furthermore, our approach extends the scope of mapping design from fixed access sequences to probabilistic memory access models, and highlights both the strengths and limitations of the BIM-based approach with respect to row miss minimization.
Similarity search forms the backbone of many clustering algorithms, requiring repeated access to large datasets in memory. In traditional von Neumann architectures, the physical separation between memory and compute units introduces significant data-movement overhead, limiting the scalability of large-scale similarity search. Content Addressable Memories (CAMs) overcome this bottleneck through parallel in-memory search, with analog CAMs further mitigating analog-to-digital conversions overhead by operating directly in the analog domain. In this work, we present a framework for fuzzy similarity search with differentiable content-addressable memory (diff-CAM). Using a 6T2M diff-CAM, with a continuous I-V response function, we demonstrate adaptive programming and training of single-point exact-match response, establishing the foundation for fully analog similarity search with weighted and fuzzy match score. To illustrate its utility, we apply the framework to clustering tasks, evaluating K-means and fuzzy C-means clustering based on diffCAM-driven similarity search. Our results achieve performance comparable to conventional similarity functions while directly exploiting hardware-efficient in-memory similarity search. By establishing a mechanism for adaptive programming and trainable exact-match response, we demonstrate the fuzzy match capability of diff-CAM. Our approach positions diff-CAMs as a key enabler for end-to-end trainable analog in-memory computation pipelines, paving the way toward scalable and energy-efficient accelerators.
Memory bottlenecks caused by contention in modern multicore systems limit scalability for many HPC and scientific workloads. To support rapid, actionable contention analysis, we introduce HOME (Hierarchy-Oriented Memory Evaluation), a lightweight framework that processes sparse PTWRITE traces by temporally interpolating sampled events (implemented as timestamp scaling) to restore the time-varying memory access pressure; HOME then applies cache-and-memory modeling and confidence conditions to flag under-sampled runs. HOME targets the middle ground between slow cycle-accurate simulators and coarse hardware-counter summaries: it preserves time-aware access intensity where it matters while running far faster than full simulation. We validate HOME on 19 applications drawn from five benchmark suites, comparing against hardware performance counters (perf), Dinero, DRAMsim3, and Sniper. HOME achieves average LLC cache contention error rates of 9.92% and memory bandwidth utilization error rates of 7.33% compared to measured hardware, while running approximately 26.8 times faster than Sniper on our GAP workloads. We also present sampling diagnostics and confidence conditions that identify workloads for which temporal interpolation is unreliable, and we provide guidance for increasing sampling density (e.g., merged runs) when needed. HOME is designed to be reproducible and practical for HPC researchers and system engineers who need quick, interpretable contention diagnostics.
Modern memory controllers use a write queue to optimistically defer DRAM write operations until a rank is idle to avoid disrupting latency-critical reads. Unfortunately, when the write queue reaches capacity without adequate idle time available, all pending read operations suffer a severe performance penalty while waiting for the memory controller to perform a mandatory bus turnaround and drain the outstanding writes. To mitigate the performance impact of write draining, we introduce Split Writes, which divide write operations into two phases: Data Transfer and Row Access. SplitWrites enable the memory controller to first transfer the write data to a small, fast buffer-called the Split Write Cache (SWC)-located within the DRAM chip during write draining. Data Transfer avoids row activation and precharge, significantly reducing the latency of write draining. Later, the memory controller can exploit bank idle time, which is more common than rank idle time, to opportunistically write data back from the Split Write Cache to DRAM rows. Unlike prior techniques that delay writes or modify cache behaviour, SplitWrites enable timely write draining while minimizing read disruption. Furthermore, Split Writes preserve protocol correctness, requiring modest architectural changes while improving system responsiveness under high write pressure.
Computing trends are leading high performance and enterprise platforms to adopt memory systems with increasingly complex architectures. As a result, many high end systems now include multiple types of memory with different capabilities and performance or distributed memory resources connected via a fast interconnect. New data management strategies are needed to exploit the unique advantages of these diverse and distributed architectures. As the primary generators of memory accesses, applications are well-suited to guide and tailor memory management for optimizing usage of these architectures. However, conventional data management in the operating system often proceeds with little knowledge of application intents or behaviors. This semantic gap limits optimization opportunities and can lead to inefficient utilization of complex memory resources. To address these challenges, this work proposes VMem: an application runtime and programming interface for enabling direct application control of physical memory resources. Designed and developed in Linux, VMem leverages standard Linux features and system calls to delegate key physical memory management tasks, including the allocation and recycling of physical memory, to the application itself. VMem does not require custom kernel code or non-standard hardware, and through integration with the memory allocator, can be deployed for use with many applications without needing to update or recompile application source. Experiments with the SPEC (R) CPU 2017 benchmarks, this work demonstrate that many applications can use VMem to exert control over physical memory resources with little or no overhead compared to the default software stack. Additionally, this work discusses opportunities to improve memory utilization with VMem and demonstrate this potential by using it to implement an optimization that uses page replication to reduce costs associated with data migration.
Modern applications, like machine learning and graph/database algorithms, demand more memory capacity and bandwidth for efficient data access. This drives advances in memory technologies, including the shift from DIMM-based systems to high bandwidth memory (HBM) in some accelerators, as well as the concept of processing in memory (PIM), which moves computation to memory. PIM boosts memory bandwidth, reduces energy usage, and preprocesses data. Recent industry products integrate PIM into HBM memory[13]. These PIM-enabled systems focus on a limited number of in-memory operations to avoid excessive overhead and we believe that enabling indirect addressing or virtual-to-physical address translation is a crucial one. Our design enhances GPUs with PIM-enabled HBM, adding indirect addressing through hardware and software co-design without requiring to perform page table walks. We achieve a general performance increase of similar to 1.4 - 1.6x for large input sizes while providing similar to 60 - 70% energy savings with minimal area overheads.
While emerging memories are considered promising technologies to replace DRAM, it has been proven difficult to penetrate the mature DRAM market and achieve profitability. Therefore, it is crucial to reduce the cost of the emerging memory media. One important way to reduce the cost is to make the memory system more fault tolerant, thus the production process can be simplified and media yield can be improved. This paper focuses on stuck failures, which are dominant in endurance failures and manufacturing failures. We present ECC Replay, an efficient and practical scheme which handles the stuck failures by utilizing existing ECC techniques in a typical emerging memory system. When a codeword read results in an uncorrectable, ECC Replay detects the stuck bits of the codeword by writing all-0/all-1 data into the media and then reading it back. ECC Replay then tries to get a correctable result by reperforming ECC correction with all the possible data combinations of the stuck bits. We also propose special techniques to minimize the Miscorrection Rate, which is often the reliability bottleneck of memory systems, and read tail latency. With minimal change in the memory controller architecture and no change to the emerging memory media, ECC Replay achieves 30x improvement of stuck-bit tolerance in a typical emerging memory system with negligible impact on memory bandwidth and read latency.
