Emerging memory interconnect (such as CXL and HCCS) promises rack-scale machine to become a reality, as the interconnect enables load/store accessible memory shared across the entire rack. However, the rack-scale shared memory poses two unique challenges on the operating system, primarily because of synchronization bottleneck and reliability issue. First, hardware cache coherence is not guaranteed, thus existing lock-based approach is ineffective to synchronize cross-node memory access. Second, memory faults significantly increase, and additional interconnect hops and switches expand fault surface and radius. As a result, current systems cannot efficiently leverage in-rack shared memory and instead manage rack resource in a disaggregated way, suffering from unnecessary networking/RDMA transmission overhead and redundant data copies. This paper proposes FlacOS, a shared operating system for memory-interconnected rack-scale architecture. FlacOS fully exploits the scalability, elasticity, and capacity advantages of rack-scale machine through shared memory. FlacOS strategically extracts and places kernel data structures in the shared memory to achieve uniform and shared operating system functionalities within the rack. FlacOS co-designs lock-free synchronization algorithms and system-wide fault tolerance mechanism to simultaneously ensure high performance and reliability. Experiments using Redis on a physical 640-core rack machine illustrate that FlacOS achieves a latency reduction of 1.75-2.4 times compared to network-based solutions.
Cache systems are widely used to speed up data retrieving. Modern HPC, data analytics, and AI/ML workloads generate vast, multi-dimensional datasets, and those data are accessed via complex queries. However, the probability of requesting the exact same data across different queries is low, leading to limited performance improvement when a traditional key-value cache is applied. In this paper, we present MosaicCache, a proactive and general caching framework that enables applications with efficient partial overlapped data reuse through novel overlap-aware cache interfaces for fast content-level reuse. The core components include a metadata manager leveraging customizable indexing for fast overlap lookups, an adaptive fetch planner for dynamic cache-tostorage decisions, and an async merger to reduce cache fragmentation and redundancy. Evaluations on real-world HPC datasets show that Mosaic-Cache improves overall performance by up to 4.1x over traditional key-value-based cache while adding minimal overhead in worst-case scenarios.
Hazard monitoring systems rely on micro datacenters (MicroDCs) for local data processing and real-time response in resource- and energy-constrained environments. These MicroDCs often host diverse, multitenant applications-such as object detection and sensor data ingestion-that contend for shared memory. Through a case study of compute-intensive and I/O-intensive applications, we show that different applications use memory differently (e.g., heap vs. OS-managed page cache), leading to asymmetric performance degradation under memory pressure. Our findings highlight the limitations of existing OS-level resource management approaches and motivate the need for cross-layered coordination between applications and the operating system to treat all memory uses as first-class citizens and adapt to changing workload demands in MicroDCs.
SPDK, as one of the most efficient I/O storage software, is capable of delivering the lowest I/O latency. Unfortunately, the polling mechanism in SPDK wastes tremendous CPU clock cycles, especially under small I/O operations and low queue depths. Although SPDK supports the conventional interrupt method, it does not improve power efficiency under such circumstances. To address this issue, we propose SPDK+, which enables the user interrupt feature in the SPDK to achieve both low latency and high power efficiency. Specifically, SPDK+ employs user interrupt handling to directly process MSI-X interrupts from SSD devices and utilizes user wait instructions during IO wait periods to conserve power. The comprehensive evaluation results show that SPDK+ achieves up to 49.5% power efficiency improvement while keeping the I/O latency almost unchanged compared with SPDK.
In-memory caches play a critical role in storage environments by reducing data access latencies and loads on backend data stores. A cache's eviction policy significantly impacts its attained miss ratio, and recent modeling techniques allow for efficient evaluation of different eviction policies at runtime. However, modern in-memory caches lack the ability to switch between eviction policies at runtime, except for Redis that can only switch between LRU and LFU. We present PaperCache, an in-memory cache capable of switching between multiple different eviction policies at runtime. Our evaluation shows that immediately after an eviction policy switch, PaperCache's behavior closely mirrors that of a cache implementing the target policy exactly (with a miss ratio typically within 1%) for a short period of time, after which PaperCache's behavior is fully inline with an exact policy implementation. Further, PaperCache is able to periodically and automatically switch to the policy exhibiting the lowest miss ratio, reducing the overall miss ratio by up to 48.5%.
The expansion of memory disaggregation, driven by datacentric applications, increases heterogeneity in memory systems. This shift enables the use of inexpensive, yet lifetimelimited, flash memory to be used as a memory expansion module. We argue that TRIM should be introduced into memory management systems to effectively respond to this transition. In this position paper, we explore the potential adoption of flash memory as memory expansion and present an analytical model that offers a straightforward yet rigorous evaluation of TRIM's effectiveness. Using this model and characteristics extracted from real-world workloads, we evaluate the effectiveness of TRIM in scalable memory systems and prove its necessity.
Disaggregated storage architectures have become a critical component in modern data centers, offering independent scaling of compute and storage. However, disaggregation introduces performance challenges, particularly due to the overhead of remote storage access. We first conduct a detailed end-to-end analysis of existing caching strategies and identify the critical issues, such as high server resource consumption, data duplication, lack of fair server resources scheduling, inefficient eviction, and prefetching policies. To address these limitations, we present the preliminary design of OrcaCache, an orchestrated, unified caching framework that coordinates between clients and storage servers. OrcaCache aims to carefully shift cache indexing to clients by exposing a global cache view, with an aim to reduce server CPU usage and duplication of data across caches. OrcaCache also aims to improve cache efficiency, adaptiveness, and fairness across servers and clients.
Efficient vector query processing is critical to enable AI applications at scale. Recent solutions struggle with growing vector datasets that exceed single-machine memory capacity, forcing unnecessary data movement and resource underutilization in monolithic architectures. We present d-HNSW, the first disaggregated vector similarity search engine for RDMA-based remote memory systems that achieves high performance while supporting fast data indexing with low network communication overhead. The core of d-HNSW is a novel disaggregation of the graph-based vector indexing data structure HNSW. It exploits the characteristics of greedy searching in HNSW to efficiently coordinate data transfers from the memory pool to the compute pool while serving data requests. Specifically, it leverages three ideas: (i) Representative index caching, a lightweight index constructed from a sampled subset of data, is cached in the compute pool to reduce frequent access to critical components of the hierarchical graph-based index, (ii) RDMA-friendly data layout design to reduce the networking round trips incurred by vector query and insertion and (iii) batched query-aware data loading to reduce bandwidth usage on data transfer between pools, addressing the limited cache capacity in compute nodes. We evaluate d-HNSW with extensive benchmarking datasets. The experimental results show that d-HNSW outperforms Naive d-HNSW implementation by up to 117x in latency while maintaining recall as 0.87 in dataset SIFT1M@1.
Recent computational storage devices enable host-side tasks such as SQL filtering and key-value operations to be offloaded to the device. However, these tasks often involve small payloads, typically a few dozen to hundreds of bytes, which are inefficiently handled by the conventional NVMe protocol due to its page-based DMA mechanism. Even tiny payloads incur 4 KB PCIe transfers, leading to severe bandwidth waste and increased latency. Prior approaches either break NVMe compatibility or are only effective for very small payloads on the order of a few dozen bytes. This paper presents ByteExpress, a new mechanism that efficiently transmits small payloads by placing them inline in 64-byte chunks directly into the NVMe submission queue, immediately following the NVMe command. ByteExpress requires only slight modifications to the NVMe driver and controller logic, while preserving full compatibility with existing APIs and SSD architectures. We implemented ByteExpress on the Linux NVMe driver and OpenSSD, demonstrating up to 98% reduction in PCIe traffic and 40% and 39% lower latency compared to PRP and a stateof-the-art approach, respectively, for sub-page payloads.
Environmental stressors such as temperature, humidity, vibration, and radiation can severely impact the performance and reliability of SSDs, particularly in edge, automotive, aerospace, and datacenter deployments. Capturing sensor data in the field and conducting accelerated lab experiments are challenging, as they are time-consuming, resource-intensive, and often destructive to hardware. Specialized setups, such as thermal chambers or vibration rigs, are also required, which is why fewstudies explore this area, and current storage management techniques like RAID, tiering, and deduplication do not consider environmental factors. Models to capture these impacts would open new research opportunities across various fields. However, accurately modeling these effects remains challenging due to,. the limited availability of experimental data,. the complex, domino-like impact of historical exposure,. the interrelated nature of environmental factors, such as temperature and humidity, which exhibit correlation,. different response of each type of NAND flash memory TLC, MLC, and SLC to environmental factors, and. the difficulty that analytical and simple machine learning models face in generalizing across devices, environments, and unseen combinations of stressors. We believe that LLMs may offer a transformative alternative to this complex problem, with embedded domain knowledge and reasoning capabilities, to facilitate prompt-based natural language interaction. We propose a hybrid framework that combines Chain-ofThought prompting and Retrieval-Augmented Generation to guide LLMs using physical principles and prior experiments. It enables interpretable "what-if" analysis of SSD behavior under environmental changes. Our results show that the LLM can effectively model the impact of temperature, humidity, and vibration on SSD performance, producing tail latency and bandwidth predictions with minimal error. The code and data are available on GitHub at https://github.com/Damrl-lab/SSD_LLM.
Hybrid cloud applications elastically burst to public clouds from an on-premise private cloud. In this setup only the public cloud directly charges applications for storing and processing data while the on-premise storage and network are already paid for and hence are considered free of charge. Consequently, application designers are naturally inclined to store and serve data remotely, while only paying for compute in the public cloud. In this work we claim, perhaps counter-intuitively, that it is often the case that hybrid cloud applications should pay for cloud storage that is co-located with the cloud computations as a mean to reduce overall costs. A co-located cloud storage can serve as a cache for frequently-accessed data and help minimize data transfers from the on-premise cloud over relatively slow inter-cloud networks. This practice can implicitly reduce the public cloud compute charges by reducing wait time of remote reads from the on-premise location, which more than compensates for the increase in cloud storage costs. We demonstrate the potential benefits of our proposed caching scheme for two elastic computing models, on-demand and serverless, and multiple cache policies that utilize a cloud-resident object storage service. Our evaluation, based on cloud object store traces, suggests that this method can achieve substantial cost reduction, at times reducing costs by up to 85%.
Emerging data-intensive applications increasingly demand large-scale, cost-effective, and high-performance memory solutions. Samsung's CXL Memory Module-Hybrid (CMM-H) uniquely integrates DDR DRAM and NAND flash storage within a single CXL-attached memory device for higher capacity while keeping still high performance. This paper provides a comprehensive exploration of the CMM-H module, detailing its architectural design, operational workflow, and caching mechanisms. We evaluate the performance of CMMH through extensive experiments, highlighting its benefits and limitations. Our study contributes insights into hybrid CXL memory architectures and provides valuable guidance for future CXL memory design improvements.
As the volume of retained data continues to increase, it is important to design primary storage systems that efficiently respond to access requests while also providing strong sustainability by reducing carbon emissions. Just over two decades ago, the Massive Arrays of Idle Disks (MAID) architecture was introduced as an energy-efficient alternative to traditional HDD-based always-on storage, employing aggressive spin-down strategies to reduce power consumption. However, high access latencies and hardware limitations led to its decline. In this work, we propose a tiered SSD+MAID storage model that combines the low-latency advantages of SSDs with the energy and carbon efficiency of a MAID system, thus offering a modern alternative to MAID while achieving lower carbon emissions than all-SSD storage. To assess the sustainability impact of such a tiered storage system, we develop a comprehensive carbon emission model that incorporates access patterns, update behaviors, and HDD spin-up dynamics. This model captures both operational and embodied carbon costs, enabling evaluations of primary storage with sustainability in mind. Through real-world workloads, we evaluate the proposed SSD+MAID system and show that it can provide a good trade-off between performance, price, and sustainability.
Databases widely adopt the technique of logging for durability and consistency, while it introduces severe performance overhead. Efforts to mitigate the logging overhead include optimizing the fsync syscall and using preallocated log files with fdatasync to persist necessary data only. Additionally, hardware advancements like power loss protection (PLP) in modern solid-state drives (SSDs) have reduced the time cost for each I/O operation at the hardware level. Yet, logging I/Os still bottleneck many databases like OceanBase due to the traversal through multiple software layers that jointly impose a significant software tax. In this paper, we find that preallocation establishes a log file's stable structure, minimizing the likelihood of space reallocations and permission changes. Leveraging this, we propose Exitos, which reorganizes the in-memory offset-to-block mapping per log file for quick lookups and creates a direct I/O path from database to SSD, bypassing most of the software layers. Implemented with eBPF on an NVMe SSD, Exitos improves the performance of OceanBase by up to 2.3x with write-intensive workloads.
In this work, we design SnapBPF, an eBPF-based snapshot prefetching mechanism, targeting VM-sandboxed serverless functions, which enables the efficient capture and prefetching of function working sets in kernel-space. SnapBPF deduplicates function working sets in memory and obviates the need for separately serializing them on disk. We complement SnapBPF with a lightweight paravirtualized interface to efficiently handle VM-sandbox memory allocations without requiring any snapshot pre-processing. Our evaluation shows that SnapBPF is able to match and improve state-ofthe-art performance with regard to i) function invocation latency and ii) memory usage for concurrent function invocations, without separately serializing working sets on disk or requiring any preemptive snapshot scanning.
Quantum Neural Networks (QNNs) harness quantum superposition and entanglement, offering promising advantages for machine learning tasks. However, noise in quantum computers frequently disrupts QNN training, wasting computational resources and extending queue times. This paper introduces the first QNN checkpointing framework to address this challenge. Through experiments on various quantum devices, we demonstrate that QNN behavior is fundamentally hardware-dependent, with the same model performing differently across platforms. This key finding shows that quantum checkpoints require additional metadata about hardware specifics and shot counts unique to quantum systems. Our framework requires minimal storage (only 186.6KB for a 100-qubit QNN) and negligible overhead, enabling frequent checkpointing to enhance training resilience and reproducibility in the NISQ era.
CPU frequency scaling is widely employed to dynamically adjust the speed and voltage of CPU cores in real time, aiming to achieve both high performance and power efficiency. Prior work indicates that variations in CPU frequency can leak information about the workload being processed. We find that the performance of I/O requests-such as file I/Os on a fast storage device-is affected by runtime changes in CPU frequency and, in turn, reflects the behavior of the ongoing workload. Accordingly, we develop IOLeak, a new side channel based on CPU frequency scaling that does not require direct access to the CPU frequency. We first construct an IOLeak covert channel by detecting I/O latency online for secretive communication, both in noisy and noise-free environments. Next, we leverage IOLeak to launch stealthy attacks, such as extracting cryptographic keys and fingerprinting websites, and confirm that IOLeak successfully leaks information through file I/Os.
Key-value stores face challenges from skewed dynamic workloads. We presents CableCache, an in-network request deduplication system. It utilizes programmable switches to maintain an object request information directory and employs the INRD protocol to deduplicate requests for hot objects with the same key, which not only overcomes the limitation of target object value size but also shortens the access path to some extent. Additionally, it can address complex scenarios such as the presence of read-modify-write requests and multi-packet target objects. Experiments show that the system effectively alleviates load imbalance in key-value stores, improves system-wide throughput, and reduces tail latency of requests under both synthetic and real workloads.
Neighborhood sampling is a critical computation step in graph learning with Graph Neural Networks (GNNs), often accounting for the majority of the training time. To mitigate this bottleneck and scale training to very large graphs, existing approaches offload the sampling computation to GPUs or computational storage, such as SmartSSDs. Given the ubiquity of multi-core CPUs and high-throughput SSDs, we investigate a simpler design that performs CPU-based sampling, making GPU resources fully available to the aggregation stage of training instead. We propose RingSampler, a new GNN sampling system that leverages io_uring to support efficient training of billion-edge graphs on a single machine. RingSampler parallelizes sampling by transparently assigning mini-batches to threads and effectively overlapping computation with I/O operations. Our results showthat RingSampler significantly outperforms SmartSSD-based sampling on large graphs and is competitive with GPU-accelerated approaches on graphs that fit in main memory.
We present ContextPrefetcher, a host-guided high-performant prefetching framework for near-storage accelerators that prefetches data blocks from storage (e.g., NAND) to device-level RAM. Efficiently prefetching data blocks to device-level RAM reduces storage access costs and improves I/O performance. We introduce a novel abstraction, Cross-layered Context (CLC), a virtual entity that spans across the host and the device and is used for identifying, managing, and tracking active and inactive data such as files, objects (within object stores), or a range of blocks. To support efficient prefetching of actively used CLCs to device memory without incurring near-device resource (memory and compute) bottlenecks, ContextPrefetcher delegates prefetching management to the host, guiding near-device compute to prefetch blocks of active CLC. Finally, ContextPrefetcher facilitates the swift reclamation of blocks associated with inactive CLC. Preliminary evaluation against state-of-the-art near-storage accelerator designs demonstrates performance gains of up to 1.34X.
