DNN model versions are used for various tasks such as fine-tuning for downstream tasks, explainability, and debugging. Numerous checkpointing solutions exist that can be adapted to persist intermediate versions of a model, as it is being trained, at different storage locations. Additionally, version management tools allow us to log, visualize, compare, and query metadata related to ML, tracking changes made to previously built models. However, the version creation process of existing methods incurs high runtime and storage overheads. In this paper, we introduce LATTICE, a low-latency, direct persistence-based DNN versioning library for Non-Volatile Memory (NVM) expansion devices. LATTICE minimizes stalls during model versioning and reduces end-to-end versioning time by reorganizing the version creation workflow, streamlining memory allocation and deallocation for efficient snapshot creation, and leveraging multi-threaded parallelism. We also develop a user-friendly versioning API that transparently implements direct persistence. Our comprehensive evaluation with diverse DNN models shows that LATTICE can reduce persistence time by as much as 99.99%, decrease end-to-end versioning time by up to 72%, reduce versioning stalls by up to 35%, and increase versioning frequency by 0.2x - 3.84x compared to state-of-the-art solutions. LATTICE also reduces space utilization for different workloads. The space savings are from 23.8% to 43.2% for workloads where model layers are progressively frozen and from 84.8% to 98.9% for fine-tuning workloads where only the last layers are tuned.
Sudden power outages remain one of the biggest threats to losing data, disrupting systems and causing financial damages. Whole system persistence (WSP) has previously been proposed as a solution to mitigate such threats through the use of non-volatile main memory (NVRAM). However, it missed out on external device state persistence and the NVRAM technology used at the time was expensive and limited with regard to their scalability. Today's NVRAM technologies are both more affordable and offer much higher storage capacities, but are typically slower than DRAM. In this paper, we show how WSP can be attained for general-purpose operating systems using modern hardware. Unlike previous attempts at realising WSP, we discuss and evaluate practical solutions that allow us to also include external devices in our considerations for practical system-wide persistence (pWSP). Despite having the OS and all applications running in slower memory, we are able to show, that specific applications that otherwise have to rely on userspace persistence mechanisms, can achieve up to 2x better throughput and up to 1.5x reduced latency, as well as significantly lower latency variance in general. We further demonstrate, that power outage-related service downtime can be reduced from potentially hours down to just minutes, allowing even network connections to continue seamlessly in many cases.
Virtual machines (VMs) impose increasing memory demands, exposing the capacity and cost limitations of traditional DRAM only memory architectures. To address this problem, heterogeneous DRAM+CXL tiered memory management systems have emerged as a promising solution. However, in virtualization environments, the semantic gap between guest and host abstraction layers, coupled with dynamic workload behaviors, hinders precise page tracking, classification, and efficient page migration across memory tiers. We introduce vtism, a virtualization-aware tiered memory management system tailored for DRAM+CXL heterogeneous memory architectures. vtism consists of three core components: (1) an adaptive guest-side page tracking mechanism that monitors memory access patterns within the guest OS and dynamically adjusts the scanning interval according to workload intensity; (2) a guest-host cooperative page classification strategy, enhanced with Multi-Generational LRU (MGLRU), which combines guest access semantics with host page states to improve the accuracy of hot/cold page identification; and (3) a NUMA-aware asynchronous page migration mechanism that maintains separate promotion and demotion queues on each NUMA node and schedules migrations based on source-destination node pairs. Our evaluation shows that vtism outperforms state-of-the-art tiered memory management systems by up to 89%, and achieves an average performance improvement of 20% across six memory intensive workloads, including graph processing, machine learning and key-value store.
Log Structured Merge Tree (LSM-Tree) is the core data structure that powers many modern key-value storage engines for its high write throughput property. To enable high speed writes, LSM-Tree ingests updates in an out-of-place manner and organizes key-value items into multiple levels of exponentially increasing capacities. To service fast data access, LSM-Tree frequently performs compaction operation, where data between two consecutive levels is sort-merged and written down to the lower level in granularity of SSTable. This compaction operation is known to cause high write amplification, which is the main threat to the LSM-tree's design objective of achieving high write performance. To tackle high write amplification of compactions in LSM-Tree, we propose a new compaction approach called Collapsed, which capitalizes on the skewed nature of key density within a level. Breaking away from the conventional approach of bounding compaction between two levels, Collapsed compaction enables data to be moved across multiple levels at once, avoiding unnecessary rewrites. Using index keys of SSTables, Collapsed compaction determines the amount of data that will be written to a level and identifies the ideal last level to which a compaction will be performed. We have implemented a prototype of Collapsed into RocksDB named CollapseDB. Experimental results show that compared to state-of-the-art LSM-trees, CollapseDB can significantly lower write amplification, up to 40% compared to RocksDB.
Emerging Persistent Memory (PMEM) applications bypass the traditional I/O stack and access physical PMEM directly to achieve high performance. However, this access model often leads to poor resource utilization and limited scalability. This paper presents Oasis, which virtualizes PMEM to provide the illusion of infinite-capacity PMEM, and enables PMEM oversubscription without requiring application changes. At the core of Oasis is a persistent mmap operation that maintains persistence and crash-consistency guarantees while automatically migrating data between PMEM and a backing store. This allows idle memory to be reclaimed for use by other applications. Oasis leverages mature memory management techniques, avoiding the need to reinvent core mechanisms while delivering strong performance benefits. Oasis is evaluated within the Linux kernel across a broad range of workloads. Oasis improves system-wide performance by up to 1.7x by oversubscribing PMEM and enabling more applications to run concurrently. Under oversubscription, Oasis reduces physical PMEM usage by up to 80% with modest overhead. Even when PMEM is not oversubscribed, Oasis delivers up to 30% performance improvement due to more efficient resource management.
