Safe kernel extensions have gained significant traction, evolving from simple packet filters to large, complex programs that customize storage, networking, and scheduling. Existing kernel extension mechanisms like eBPF rely on in-kernel verifiers to ensure the safety of kernel extensions by static verification using symbolic execution. We identify significant usability issues—safe extensions being rejected by the verifier—due to the language-verifier gap , a mismatch between developers’ expectation of program safety provided by a contract with the programming language, and the verifier’s expectation. We present Rex, a new kernel extension framework that closes the language-verifier gap and improves the usability of kernel extensions in terms of programming experience and maintainability. Rex builds upon language-based safety to provide safety properties desired by kernel extensions, along with a lightweight extralingual runtime for properties that are unsuitable for static analysis, including safe exception handling, stack safety, and termination. With Rex, kernel extensions are written in safe Rust and interact with the kernel via a safe interface provided by Rex’s kernel crate. No separate static verification is needed. Rex addresses usability issues of eBPF kernel extensions without compromising performance.
ASPLOS 2024 received a record of 922 submissions (spread across three review cycles), from which 194 papers were accepted. This record represents an increase of about 50% over 2023, which itself saw a similar increase over 2022. As a result, selecting the best papers proved especially challenging. In this document, we describe the methodology we employed—a process we believe is fair, upholds high standards of quality, and emphasizes consensus and expert judgment. We hope it will be useful for others.
Caches reduce latency and network traffic, and cache performance largely depends on the eviction policy. Eviction effectiveness is typically measured by byte and object miss ratios. Although learning-based eviction policies can reduce misses, their high computational overhead limits practical deployment. This work presents 3L-Cache+, an extended and refined version of 3L-Cache. 3L-Cache+ is an object-level learning-based eviction policy that achieves low computational overhead while delivering strong overall miss-ratio performance. To reduce overhead, we introduce two key techniques. First, we design an efficient training-data collection scheme that filters redundant requests and dynamically adjusts the training frequency. Second, we propose a lightweight eviction method that combines bidirectional sampling, which prioritizes unpopular objects, with an efficient eviction-candidate selection strategy. In addition, an auto-tuning mechanism improves adaptability across traces. We evaluate 3L-Cache+ in a testbed using 4,855 traces. The results show that 3L-Cache+ reduces average CPU overhead by 74.7% and 95.5% compared with HALP and LRB, respectively, and by 22.7% compared with the prior 3L-Cache baseline. 3L-Cache+ incurs only 4.8× LRU overhead under small cache sizes and 3.1× under large cache sizes, while achieving the best byte- or object-miss-ratio performance among 11 state-of-the-art policies.
Coverage-guided kernel fuzzing is an effective technique for ensuring the reliability and security of operating system (OS) kernels. For example, Syzkaller, a widely used and representative coverage-guided kernel fuzzer, has successfully uncovered over 5000 bugs in Linux to date. Specifically, in kernel fuzzing, input minimization is a critical stage for improving fuzzing efficacy by compressing interesting inputs (programs) into shorter and coverage-preserving seeds. If this minimization stage is omitted, the achieved coverage and number of unique bugs found in Syzkaller would decrease by 27.5% and 40.4%, respectively, during a 48-hour fuzzing campaign. However, the minimization stage also takes over half of all fuzzing resources. During the 48-hour fuzzing campaign, 57.5% of program executions are expended in this stage, significantly limiting the potential of kernel fuzzing. To the best of our knowledge, no prior work has explored or mitigated the problem described above in kernel fuzzing. To bridge this gap, this paper introduces two general and novel optimization strategies — influence-guided call removal and type-informed argument simplification — to reduce the cost of the minimization stage. The key idea of these two strategies is to reduce the number of program executions required to verify whether the new coverage achieved by the inputs is preserved. We built SyzMini, a prototype based on Syzkaller that implements our two strategies to optimize input minimization. The evaluation shows that SyzMini significantly reduces the cost of the minimization stage by 60.7%, leading to a 12.5% improvement in branch coverage and a 1.7~2X increase in the number of unique bugs found. In addition, SyzMini has found 13 previously unknown bugs in the latest upstream kernel, all of which have been confirmed, with four already fixed. In terms of applicability, our optimization strategies can also improve the effectiveness of other kernel fuzzers ( e.g. , SyzVegas, SyzDirect, and CountDown). Additionally, based on the insight that calls with indirect influence on the target call have a low probability of actually affecting its execution, we extended the influence-guided call removal strategy by introducing an additional attempt. This attempt preserves only the direct influences to further reduce the cost during call removal. We also integrated the extended strategy into SyzMini, which we refer to as SyzMini + . Our evaluation results demonstrate that SyzMini + achieves a coverage improvement of 14.5% to 16.5% and uncovers 1.9 to 2.3 times as many unique bugs as Syzkaller, further enhancing the effectiveness of SyzMini. We have released the implementations of both SyzMini and SyzMini + as open-source projects, which is available at [1].
Fail-slow hardwares are still running and functional, but in a degraded mode, thus slower than their expected performance. Bugs triggered by fail-slow hardwares cause severe cloud system failures. Existing testing tools fail to efficiently detect these bugs due to overlooking their characteristics. In order to address this problem, this paper provides a bug study that analyzes 48 real-world fail-slow hardware failures from typical cloud systems. We observe that (1) fail-slow hardwares make high-level software components vulnerable, including synchronized and timeout mechanisms; (2) the fine granularity of fail-slow hardwares is necessary to trigger these bugs. Based on these two observations, we propose Sieve, a fault injection testing framework for detecting fail-slow hardware failure bugs. Sieve statically analyzes target system codes to identify synchronized and timeout-protected I/O operations as candidate fault points and instruments hooks before candidate fault points to enable fail-slow hardware injection. To efficiently explore candidate fault points, Sieve adopts grouping and context-sensitive injection strategies. We have applied Sieve to three widely deployed cloud systems, i.e., ZooKeeper, Kafka, and HDFS. Sieve has detected six unknown bugs, two of which have been confirmed.
We present MeshTest, the first end-to-end testing framework for traffic management of service mesh. The key idea of MeshTest is to automatically generate input configurations with end-to-end service flows, and then create real test request suites for each input based on detailed service flow modeling. There are two technical challenges. First, the input space of service mesh configurations is large and complex. The input configurations should be carefully orchestrated to form end-to-end service flow paths. Second, the abstract output network behavior cannot be directly checked for correctness, and we need to generate a set of real requests that are capable of checking possible behaviors. To address these challenges, we model the service flows of traffic management in service mesh, and propose a novel Service Flow Exploration technique to enumerate all possible configuration resources and interactions between them in the input configuration. We design and implement MeshTest, which contains an automatic input configuration generator based on Service Flow Exploration and a Service Mesh Oracle which leverages formal methods to generate test request suites. MeshTest has found 25 new bugs (20 confirmed and 10 fixed) in three popular service mesh systems, Istio, Linkerd and Cilium.
Transactions are a key feature of database systems and isolation levels specify the behavior of concurrently executing transactions. Ensuring the correctness of isolation levels is crucial. Recently, many isolation anomalies have been found in production database systems. Checkers can be used to validate that a particular execution history conforms to a desired isolation level. However, state-of-the-art checkers cannot handle predicate operations, which are both common in real-world workloads and essential for distinguishing between the repeatable read and serializable isolation levels. In this work, we address this issue by proposing two techniques for efficient white-box checking. Our key idea is to use information, that is, easily provided by database systems to efficiently check the isolation level of a given execution history. We present the version certificate recovery technique and its associated checker Emme . Version certificate recovery is a method of recovering the version order and each operation’s version set from the database system under test. To minimise execution time, we also propose the expected serialization order technique, along with its associated checker Enne , which obviates the need to define and recover a version certificate for many serializable concurrency control protocols. We have implemented version certificate recovery for three widely used database systems—PostgreSQL, CockroachDB, and TiDB. We demonstrate that Emme is 1.2–3.6× faster than Elle, a state-of-the-art checker. When paired with the expected serialization order technique, Enne obtains a further speedup of 34–430× when checking execution histories containing predicate operations. We show that our approaches can identify invalid execution histories that cannot be detected by Elle and also show that they can find realistic bugs purposely introduced by an engineer. Finally, we create a new checker, King Cobra , by modifying Cobra, an existing black-box checker, in order to conduct an ablation study to evaluate how the performance of a black-box checker changes when provided with varying degrees of ordering information. By doing so, we highlight the fundamental role that ordering information plays in the execution time of an isolation level checker and explore the limits that this places on a truly black-box checker.
The OS page cache is central to the performance of many applications, by reducing excessive accesses to storage. However, its one-size-fits-all eviction policy performs poorly in many workloads. While the systems community has experimented with a plethora of new and adaptive eviction policies in non-OS settings (e.g., key-value stores, CDNs), it is very difficult to implement such policies in the page cache, due to the complexity of modifying kernel code. To address these shortcomings, we design a flexible eBPF-based framework for the Linux page cache, called cache_ext , that allows developers to customize the page cache without modifying the kernel. cache_ext enables applications to customize the page cache policy for their specific needs, while also ensuring that different applications’ policies do not interfere with each other. Additionally, we introduce an observability tool, cachestream , which enables fine-grained tracing of the page cache, allowing precise refinement of applications and cache_ext policies. We demonstrate the flexibility of cache_ext ’s interface by using it to implement eight different policies. Our evaluation shows that it is indeed beneficial for applications to customize the page cache to match their workloads’ unique properties, and that they can achieve up to 70% higher throughput and 58% lower tail latency.
Extended user interrupts (xUI) is a set of processor extensions that enable near zero-cost notification in user space. This paper deconstructs Intel’s UIPI design, the only existing user-space interrupt system, through analysis and measurement, and uses this to develop an accurate model of its timing. It then introduces five novel enhancements to user interrupts: Tracked interrupts, hardware safepoints, a kernel bypass timer, interrupt forwarding, and Hardware Barriers. The paper also proposes a user interrupt system for the RISC-V architecture and shows how xUI’s enhancements can be applied to the RISC-V interrupt architecture. xUI is modeled in gem5 simulation and evaluated on four use cases: preemption in a high-performance user-level runtime, IO notification in a layer3 router using DPDK, IO notification in a synthetic workload with a streaming accelerator modeled after Intel’s Data Streaming Accelerator, and synchronization in a synthetic multi-accelerator workload. This work shows that xUI offers the performance of shared-memory polling with the efficiency of asynchronous notification.
Merkle hash trees are the standard method to protect the integrity and freshness of stored data. However, hash trees introduce additional compute and I/O costs on the I/O critical path, and prior efforts have not fully characterized these costs. In this paper, we quantify performance overheads of storage-level hash trees in realistic settings. We identify that hashing (CPU) costs are the primary performance bottleneck, and develop an analytical model demonstrating why this occurs. We then design an optimized tree structure called Dynamic Merkle Trees (DMTs) that exploits patterns in workloads to reduce tree traversal costs. We implement DMTs in a block device driver and a file system and through extensive evaluation show that DMTs can exploit patterns in workloads to deliver up to a 2.2 × throughput and latency improvement over the state of the art. Our novel approach provides a promising new direction to achieve integrity guarantees in storage efficiently and at scale.
The rapid proliferation of video services has created a strong demand for efficient content delivery. Traditional solutions primarily rely on Content Delivery Networks (CDNs), which, while effective, impose substantial bandwidth costs on video providers. To address this challenge, cost-efficient edge resources have emerged as a promising alternative. However, their heterogeneous hardware and inherently limited performance pose significant challenges for practical deployment. In this article, we present how ByteDance explores the use of these cost-efficient but less performant resources. Specifically, we first present an extensive overview of PCDN (Peer-to-Peer CDN), ByteDance’s alternative delivery network for CDNs. Second, as PCDN encounters speed degradation and resource imbalances after years of deployment, we further introduce PCDN + , the enhanced iteration of PCDN. Specifically, we first propose a joint control protocol. By using de-synchronized probing and proactive retransmission, our protocol significantly improves the performance degradation caused by self-competition and head-of-line blocking (HoL). We then introduce a resource management framework. By integrating a well-designed centralized/decentralized framework, we evolve previous “static” and “uncontrolled” PCDN into a “dynamic” and “controlled” system. The extensive A/B test and real-world deployment have demonstrated that PCDN + (1) provides higher service speed, (2) effectively alleviates overloading issues, (3) significantly improves the utilization of low-cost resources.
Current serverless platforms struggle to optimize resource utilization for both CPU and GPU functions due to their dynamic and fine-grained nature. Conventional techniques like overcommitment and autoscaling fall short, often sacrificing utilization for practicability or incurring performance trade-offs. Overcommitment requires predicting performance to prevent QoS violation, introducing trade-off between prediction accuracy and overheads. Autoscaling requires scaling instances in response to load fluctuations quickly to reduce resource wastage, but more frequent scaling also leads to more cold start overheads. The rich concurrency of GPU resources further complicates GPU instance orchestration, such as setting right batch sizes. This paper introduces Jiagu to harmonize efficiency with practicability through the following novel techniques. First, pre-decision scheduling achieves accurate prediction while eliminating overheads by decoupling prediction and scheduling. Second, dual-staged scaling achieves frequent adjustment of instances with minimum overhead. Third, Jiagu conducts an in-depth analysis about the complexity of the relationship between GPU function configuration and execution. It then proposes batch-aware scaling that achieves optimal configurations for both batch size setting and autoscaling, addressing all the challenges according to the analysis. We have implemented a prototype and evaluated it using real-world applications and traces from the public cloud platform. Our evaluation shows an improvement in deployment density over commercial clouds (with Kubernetes) while maintaining QoS for both CPU and GPU functions (54.8% and 18% respectively), and 81.0%–93.7% lower scheduling costs and a 57.4%–69.3% reduction in cold start latency compared to existing QoS-aware schedulers.
Intrusion Detection Systems (IDSes) are widely employed to identify potential attacks in guest virtual machines (VMs). Nonetheless, traditional IDSes fall short of the demands of high-performance clouds. First, monitoring VM events increases the tail latency of guest services. Second, the throughput of traditional IDSes cannot meet high-performance cloud requirements, leading to event loss and reduced detection accuracy. Finally, cloud providers typically run complex IDS tools within the VM. Updating IDS functionality requires modifying guest VMs, which hurts maintainability. To overcome these challenges, this paper presents EIDS, a cloud IDS framework with high performance and good maintainability. We observe that the main bottleneck is collecting VM status, and the collected status can be divided into fundamental and supplementary status. EIDS then splits the status collection procedure spatially and temporally. First, we provide a status monitor with a separate architecture that isolates the status collection logic in a microVM, thus minimizing the code in guest VMs and improving maintainability. Second, EIDS introduces a two-phase status collection method to handle multiple events in batches, asynchronously, for high IDS throughput. A tiny tracer, implemented with eBPF, operates inside the user VM to collect fundamental status. The complex status collector runs in an isolated microVM. It utilizes Virtual Machine Introspection (VMI) to gather supplementary status, using the fundamental status to bridge the semantic gap. The status collector batches the collection for multiple events to amortize the fixed overhead of microVM switching and improve event tracing throughput. Finally, to minimize tail latency overhead, a fine-grained and workload-aware scheduler executes IDS logic with small time slices during user VM idle periods. We implemented a prototype of EIDS in Linux-KVM and conducted a comprehensive evaluation. We compared EIDS’s performance with Falco, an open-source IDS widely used by Kubernetes and AWS for runtime security monitoring. The results demonstrate that, compared to Falco, EIDS reduces the 99 th -percentile latency overhead by 97% and achieves a 13.8X improvement in IDS event handling throughput.
Linux, developed over the past three decades, has become the cornerstone of modern computing, serving as the foundation for a wide array of applications ranging from large-scale supercomputers to resource-constrained embedded devices. However, this infrastructure has long been plagued by memory and concurrency bugs, primarily due to the permissive nature of memory operations in the C language. The Rust-for-Linux ( RFL ) project offers a promising solution to address these long-standing safety concerns by integrating Rust’s advanced features, such as static ownership and type-checking, into the kernel. This integration could potentially eliminate memory and concurrency bugs while maintaining performance. Although RFL has matured and been merged into the Linux mainline, it remains understudied, and its effectiveness in balancing safety and performance is yet to be fully assessed. In this study, we present the first empirical analysis of RFL to evaluate its current state and benefits, focusing on how Rust integrates with Linux and whether this fusion ensures driver safety without significant overhead. We analyze eight RFL drivers, examining hundreds of issues and pull requests, thousands of GitHub commits, mail exchanges from the Linux mailing list, and over 12,000 discussions on Zulip. Our findings indicate that while Rust mitigates kernel vulnerabilities, it does not completely eliminate them. If not properly managed, Rust’s safety features even introduce notable runtime overhead and increased development complexity.
Many intelligent applications, such as autonomous driving and virtual reality, require running both latency-critical (real-time) and best-effort deep neural network (DNN) inference tasks to achieve both real-time and work-conserving on the GPU. However, commodity GPUs lack efficient preemptive scheduling support, and existing state-of-the-art approaches either have to monopolize GPU or let real-time tasks to wait for best-effort tasks to complete, resulting in low utilization, high latency, or both. This article presents REEF, the first GPU-accelerated DNN inference serving system that achieves low-latency and work-conserving for concurrent real-time and best-effort tasks. REEF accomplishes this by enabling microsecond-scale kernel preemption and controlled concurrent execution in GPU scheduling. REEF is novel in two ways. First, based on the observation that DNN inference kernels are mostly idempotent, REEF devises a reset-based preemption scheme that launches a real-time kernel on the GPU by proactively killing and restoring best-effort kernels at microsecond-scale. Second, since DNN inference kernels have varied parallelism and predictable latency, REEF proposes a dynamic kernel padding mechanism that dynamically pads the real-time kernel with appropriate best-effort kernels to fully utilize the GPU with negligible overhead. Evaluation using a new DNN inference serving benchmark (DISB) with diverse workloads and a real-world trace on both NVIDIA and AMD GPUs shows that REEF only incurs less than 5% overhead in end-to-end latency for real-time tasks but increases the overall throughput by up to 1.53 & times;, compared to scheduling tasks sequentially. To demonstrate the practical benefits of our approach, we compare REEF with Triton, a widely-adopted production-level serving system. Our evaluation shows that REEF outperforms Triton by 1.12 & times; to 5.20 & times; in end-to-end latency for real-time tasks, while maintaining comparable throughput.
Huge pages are effective in reducing address translation overhead under virtualization. However, huge pages can lead to the memory bloat problem, which manifests in two primary forms: hot bloat and usage bloat . Hot bloat occurs when accesses to a huge page are heavily skewed towards a small subset of base pages, leading the hypervisor to (mistakenly) classify the entire huge page as hot. Hot bloat undermines several critical virtualization techniques, including tiered memory and page sharing. Usage bloatrefers to the base pages within a huge page that has not yet been allocated, causing virtual machines (VMs) to demand excessive memory. Prior work addressing memory bloat either requires hardware modification or targets a specific scenario and is not applicable to a hypervisor. This article presents HUGESCOPE , a lightweight, effective and generic system that addresses the memory bloat problem under virtualization based on commodity hardware. HUGESCOPE includes an efficient and precise page tracking mechanism, leveraging the other level of indirect memory translation in the hypervisor. HUGESCOPE provides a generic framework to support page splitting and coalescing policies, considering the memory pressure, as well as the recency, frequency, and skewness of page access. Moreover, HUGESCOPE is general and modular. It can not only be easily applied to various scenarios concerning hot bloat , including tiered memory management (HS-TMM ) and page sharing (HS-SHARE ), but also seamlessly expose its capabilities to VMs to address the usage bloat problem (HS-HP ). Evaluation shows that HUGESCOPE incurs less than 4% overhead, by addressing hot bloat , HS-TMM improves performance by up to 61% over vTMM while HS-SHARE saves 41% more memory than Ingens while offering comparable performance, and By addressing usage bloat, HS-HP can eliminate excessive memory usage, and achieve performance improvements of up to 11% over HawkEye.
This article presents UGACHE, a unified multi-GPU cache system designed for embedding-based deep learning (EmbDL). UGACHE is primarily motivated by the unique characteristics of EmbDL applications, namely readonly and skewed embedding accesses with affinity and predictability. UGACHE introduces a novel factored extraction mechanism that avoids bandwidth congestion to fully exploit high-speed cross-GPU interconnects (e.g., NVLink and NVSwitch). Based on a hotness metric, UGACHE also provides a near-optimal cache policy that balances local and remote access to minimize the extraction time for diverse GPU interconnect topologies. We have implemented UGACHE and integrated it into two representative frameworks, TensorFlow and PyTorch. Evaluation using two typical types of EmbDL applications, namely graph neural network (GNN) training and deep learning recommendation (DLR) inference, shows that UGACHE outperforms state-of-the-art replication and partition designs by an average of 1.93 & times; and 1.63 & times; (up to 5.25 & times; and 3.45 & times;), respectively. Furthermore, we demonstrate the applicability of UGACHE's principle beyond embedding-based deep learning, with an example of text-to-image generation on an inference cluster.
