DNN model versions are used for various tasks such as fine-tuning for downstream tasks, explainability, and debugging. Numerous checkpointing solutions exist that can be adapted to persist intermediate versions of a model, as it is being trained, at different storage locations. Additionally, version management tools allow us to log, visualize, compare, and query metadata related to ML, tracking changes made to previously built models. However, the version creation process of existing methods incurs high runtime and storage overheads. In this paper, we introduce LATTICE, a low-latency, direct persistence-based DNN versioning library for Non-Volatile Memory (NVM) expansion devices. LATTICE minimizes stalls during model versioning and reduces end-to-end versioning time by reorganizing the version creation workflow, streamlining memory allocation and deallocation for efficient snapshot creation, and leveraging multi-threaded parallelism. We also develop a user-friendly versioning API that transparently implements direct persistence. Our comprehensive evaluation with diverse DNN models shows that LATTICE can reduce persistence time by as much as 99.99%, decrease end-to-end versioning time by up to 72%, reduce versioning stalls by up to 35%, and increase versioning frequency by 0.2x - 3.84x compared to state-of-the-art solutions. LATTICE also reduces space utilization for different workloads. The space savings are from 23.8% to 43.2% for workloads where model layers are progressively frozen and from 84.8% to 98.9% for fine-tuning workloads where only the last layers are tuned.
Sudden power outages remain one of the biggest threats to losing data, disrupting systems and causing financial damages. Whole system persistence (WSP) has previously been proposed as a solution to mitigate such threats through the use of non-volatile main memory (NVRAM). However, it missed out on external device state persistence and the NVRAM technology used at the time was expensive and limited with regard to their scalability. Today's NVRAM technologies are both more affordable and offer much higher storage capacities, but are typically slower than DRAM. In this paper, we show how WSP can be attained for general-purpose operating systems using modern hardware. Unlike previous attempts at realising WSP, we discuss and evaluate practical solutions that allow us to also include external devices in our considerations for practical system-wide persistence (pWSP). Despite having the OS and all applications running in slower memory, we are able to show, that specific applications that otherwise have to rely on userspace persistence mechanisms, can achieve up to 2x better throughput and up to 1.5x reduced latency, as well as significantly lower latency variance in general. We further demonstrate, that power outage-related service downtime can be reduced from potentially hours down to just minutes, allowing even network connections to continue seamlessly in many cases.
Virtual machines (VMs) impose increasing memory demands, exposing the capacity and cost limitations of traditional DRAM only memory architectures. To address this problem, heterogeneous DRAM+CXL tiered memory management systems have emerged as a promising solution. However, in virtualization environments, the semantic gap between guest and host abstraction layers, coupled with dynamic workload behaviors, hinders precise page tracking, classification, and efficient page migration across memory tiers. We introduce vtism, a virtualization-aware tiered memory management system tailored for DRAM+CXL heterogeneous memory architectures. vtism consists of three core components: (1) an adaptive guest-side page tracking mechanism that monitors memory access patterns within the guest OS and dynamically adjusts the scanning interval according to workload intensity; (2) a guest-host cooperative page classification strategy, enhanced with Multi-Generational LRU (MGLRU), which combines guest access semantics with host page states to improve the accuracy of hot/cold page identification; and (3) a NUMA-aware asynchronous page migration mechanism that maintains separate promotion and demotion queues on each NUMA node and schedules migrations based on source-destination node pairs. Our evaluation shows that vtism outperforms state-of-the-art tiered memory management systems by up to 89%, and achieves an average performance improvement of 20% across six memory intensive workloads, including graph processing, machine learning and key-value store.
Log Structured Merge Tree (LSM-Tree) is the core data structure that powers many modern key-value storage engines for its high write throughput property. To enable high speed writes, LSM-Tree ingests updates in an out-of-place manner and organizes key-value items into multiple levels of exponentially increasing capacities. To service fast data access, LSM-Tree frequently performs compaction operation, where data between two consecutive levels is sort-merged and written down to the lower level in granularity of SSTable. This compaction operation is known to cause high write amplification, which is the main threat to the LSM-tree's design objective of achieving high write performance. To tackle high write amplification of compactions in LSM-Tree, we propose a new compaction approach called Collapsed, which capitalizes on the skewed nature of key density within a level. Breaking away from the conventional approach of bounding compaction between two levels, Collapsed compaction enables data to be moved across multiple levels at once, avoiding unnecessary rewrites. Using index keys of SSTables, Collapsed compaction determines the amount of data that will be written to a level and identifies the ideal last level to which a compaction will be performed. We have implemented a prototype of Collapsed into RocksDB named CollapseDB. Experimental results show that compared to state-of-the-art LSM-trees, CollapseDB can significantly lower write amplification, up to 40% compared to RocksDB.
Emerging Persistent Memory (PMEM) applications bypass the traditional I/O stack and access physical PMEM directly to achieve high performance. However, this access model often leads to poor resource utilization and limited scalability. This paper presents Oasis, which virtualizes PMEM to provide the illusion of infinite-capacity PMEM, and enables PMEM oversubscription without requiring application changes. At the core of Oasis is a persistent mmap operation that maintains persistence and crash-consistency guarantees while automatically migrating data between PMEM and a backing store. This allows idle memory to be reclaimed for use by other applications. Oasis leverages mature memory management techniques, avoiding the need to reinvent core mechanisms while delivering strong performance benefits. Oasis is evaluated within the Linux kernel across a broad range of workloads. Oasis improves system-wide performance by up to 1.7x by oversubscribing PMEM and enabling more applications to run concurrently. Under oversubscription, Oasis reduces physical PMEM usage by up to 80% with modest overhead. Even when PMEM is not oversubscribed, Oasis delivers up to 30% performance improvement due to more efficient resource management.
AI development faces challenges from fragmented workflows and varied LLM integrations. This modular GitHub framework offers structured guidance, seamless support for major LLMs, and clear, low-maintenance workflows---enabling sustainable and collaborative innovation.
Effective file system testing relies on coverage to detect bugs and enhance reliability. We analyzed real file system bugs and found a weak correlation between code coverage, the most commonly used metric, and test effectiveness; many bugs were in covered code but remained undetected. Our study also showed that covering diverse file system inputs and outputs---system call arguments and return values---can be key to detecting the majority of observed bugs. We present input coverage and output coverage as new metrics for evaluating and improving file system testing, and have developed the IOCov framework for computing these metrics. Unlike existing system call tracers, IOCov computes coverage using only the calls relevant to testing, excluding unrelated ones that should not be counted. To demonstrate IOCov's utility, we used it to extend the existing testing tool CrashMonkey into CM-IOCov, which achieves broader input coverage and more thorough detection of crash consistency bugs. Our experimental evaluation shows that IOCov computes input and output coverage accurately with minimal overhead. IOCov is applicable to different types of file system testing and can provide insights for improvement as well as identify untested cases based on coverage results. Moreover, the bugs found exclusively by CM-IOCov are 2.1 and 12.9 times more than those found exclusively by CrashMonkey on the 6.12 and 5.6 kernels, respectively, demonstrating the effectiveness of the IOCov-based coverage approach.
Approximately 2.2 billion people around the world live with some degree of visual impairment. For individuals with severe vision loss, maintaining functional independence and safely navigating their environment is a significant challenge [3]. In this paper, we present SafeStep, a novel wearable system that enables early detection of obstacles at waist height and above, thus filling the gap in existing aids and enhancing user safety and independence.
Querying data stored in cloud object stores often leads to network bottlenecks, particularly when large datasets need to be transferred over wide area networks (WANs) for processing. Encryption further complicates this challenge by requiring entire encrypted objects to be fetched from the object store before analysis. To address this, we push down filtering and perform secure computing near storage using a Data Processing Unit (DPU) integrated into the cloud server. We present DPUF, a DPU-assisted near-storage secure data filtering system that accelerates filtering operations by performing the query near the data and returning only the results of the query. By using the DPU as a secure enclave dedicated to and solely trusted by the client, DPUF provides a secure means of performing filtering of encrypted data near the shared (i.e., untrusted) storage system. Furthermore, our approach leverages on-board DPU accelerators and compute resources to maximize performance. On average, DPUF achieves up to 19.7x speedup over traditional client-side filtering and reduces networking costs by up to 16x.
Modern object stores expose only coarse, static configurations (replication factor, ACLs, bucket lifecycles) and must be over-provisioned to absorb demand spikes or new security requirements. MictlanX upgrades the storage layer itself with two code-defined programming models: a Responsive-Deployment Model, that lets operators declare elastic regions in YAML/Python, and an Adaptive Data-Placement Model, that drives per-object filters and dynamic replication. A 16-node prototype sustains 20 MB/s under a 0.01s burst while keeping 90% of requests below 0.1s---up to 4× faster than fixed-replica modes---and beats MinIO, Google Drive, and Dropbox by up to 23% throughput with sub-second latencies.
The growing computational demands and performance limits of AI models are often constrained by their underlying mathematical processing units. This issue spans various model sizes and hardware---from laptop CPUs to large GPU clusters. Current units, like a CPU's Floating-Point Unit (FPU), use multi-cycle pipelines. While this design supports complex computations, it also adds latency and slows floating-point performance. Spending multiple cycles on an operation is slow, but preferable to lowering the global clock frequency to support these intricate units. This paper proposes a new direction for AI and mathematical compute architecture. We leverage recent advancements, in which photonic and electronic gates can coexist on the same die [2], and mix fast photonic FPUs within the cheaper electronic logic used for the rest of the chip.
We present TieredKV, a novel tiered key-value store that seamlessly integrates a Log-Structured Merge (LSM) tree with a Learned Index to achieve superior read and write performance on storage systems. While existing approaches use learned indexes primarily as auxiliary components within LSM trees, TieredKV employs a two-tier design where the LSM tree handles recent write operations while a separate Learned Index accelerates read performance. Our design includes a non-blocking conversion mechanism that efficiently transforms LSM data into a Learned Index during garbage collection, maintaining high performance without interrupting operations. TieredKV dramatically reduces LSM size through this tiered approach, leading to significant performance gains in both reads and writes. Extensive evaluations across diverse workloads show that TieredKV outperforms state-of-the-art LSM-based solutions by up to 4.32x for read operations and 1.43x for writes, and achieves up to 7.9x better overall performance compared with on-storage Learned Index. The system demonstrates robust performance across different data distributions, access patterns, and storage media including both SSDs and HDDs.
Processing-in-Memory (PIM) helps reduce DRAM access bottlenecks in accelerating Deep Neural Networks (DNNs). Industrial solutions like SK Hynix GDDR6-AiM [1] place processing cores (PIMcores) near DRAM banks to lower data movement and latency. However, current near-bank PIM designs face two key limitations: (1) PIMcores mainly support multiply-accumulate (MAC) operations, while other tasks are handled by the host CPU or GPU, causing heavy host-PIM traffic. (2) The buffer capacity is limited (e.g., 2KB global buffer per channel [1]), reducing opportunities for data reuse.
Distinguishing actionable anomalies from noise in monitoring metrics is a critical operational challenge. In our experience, static thresholds often resulted in overwhelming alert fatigue, while pilot projects utilizing ML-based models proved to be uninterpretable. We implemented an interpretable system based on simple rules co-designed with our support team, focusing on sustained, meaningful deviations from a seasonal baseline. The result was a dramatic shift in operator trust, with feedback indicating that "all alerts are good".
In this paper, we present Pangolin, a comprehensive testing framework for configuration-rich key-value stores. To better understand bugs in modern key-value stores and explore domain knowledge for efficiently identifying new ones, we first comprehensively study historical bugs in five mature key-value stores during the last eight years. Then, we design and implement Pangolin, which is motivated by insights from our bug study, which indicated most bugs could be identified by systematically testing a small sequence of operations and configurations. Specifically, Pangolin practices these insights by introducing a bounded testing strategy into a spectrum of black-box and fuzzing test procedures. Finally, we utilize Pangolin to find 20 bugs and reproduce 443 historical bugs in five mature key-value stores (RocksDB, LevelDB, HyperlevelDB, BadgerDB, and Redis), making it an attractive supplement to handwritten test suites.
The NVMe Key Value (NVMe-KV) Command Set has been standardized to enable access to an NVMe device with a key rather than a block address and make an NVMe device a KV service provider. This new interface opens an exciting opportunity of offloading extensive data management chores to an external KV device and streamlining the KV-based data processing at the host. However, the interface itself may become a major performance bottleneck with small KV access and make the technology hard to be deployed in diverse application scenarios. In this paper we proposed a software-based facility, named SAKER, at the host side to remove or alleviate the performance bottleneck at the interface. SAKER, which was prototyped in an NVMe-KV SSD emulator, demonstrates that it can effectively keep the NVMe-KV interface from becoming the performance bottleneck even with small KV requests in most workloads.
Generating SVGs from raster images is a challenging compression task due to the semantic and structural mismatch between pixel and vector formats.We introduce VisVec, a largescale dataset of triplets: raster image, SVG vector, and textual description, designed to improve training for vision-language models (VLMs) in vector generation tasks. The initial dataset comprises 2.5k high-resolution, semantically rich examples. Our dataset aims to address key limitations in models like GPT-4V and Claude 3, which often fail to produce valid SVG outputs due to lack of structured training data. Our dataset achieves 10.52 compression ratio when using SVG over PNG. Training LLMs on it could enable high-quality image-to-SVG compression.
With the advent of Compute Express Link (CXL), hardware-managed memory tiering has become a reality. In this paper, we investigate Samsung's CXL Memory Module-Hybrid (CMM-H), a CXL Type 3 device integrating DRAM and NAND flash managed by an FPGA-based controller and providing byte-addressable memory interface via the cxl.mem protocol. We perform a detailed evaluation of CMM-H and compare its performance with OS-level and block-level tiering solutions. Our results highlight the performance benefits of CMM-H for cache-hit scenarios and identify key limitations for cache-miss situations, offering insights into the trade-offs involved in adopting hardware-managed memory tiering in emerging CXL-based systems.
Passive video consumption is a primary cause of student disengagement in online education. To address this, we developed FocusFlow, a web platform that transforms video lectures into an interactive experience. FocusFlow uses AI to monitor student engagement in real-time and intervenes with context-aware, AI-generated quizzes when it detects a drop in focus. The effectiveness of such a system hinges on its ability to provide immediate feedback, which poses a significant systems challenge. This paper presents the FocusFlow system, its features, and the critical architectural decision-offloading inference to the client-that makes its real-time capabilities possible.
Linux kernel VPNs suffer from severe performance degradation under high load due to execution order inversion (EoI), a phenomenon where packet recombination functions preempt earlier pipeline stages. This leads to severe latency spikes and throughput reductions. We investigate kernel threads and workqueues as alternative kernel asynchronous APIs to address these limitations, achieving up to a 4.7x increase in throughput while reducing tail latency by 65%. These results demonstrate the importance of selecting appropriate kernel asynchronous APIs for kernel-level network applications.
