Accurate prediction of training time in distributed deep learning is crucial for resource allocation, cost estimation, and job scheduling. We observe that the floating-point precision setting is a key determinant of training time, leading to training time variations of 2.4x over its minimum. However, existing studies on distributed training time prediction rely on static model computation graphs that do not capture precision variations, including mixed precision. According to our experiments, training time prediction without considering precision results in significant prediction errors - reaching up to 147.85

In virtualized cloud environments, meeting the network bandwidth requirements of virtual machines (VMs) is challenging due to the lack of accurate methods for translating target network bandwidth into precise CPU allocations. This translation is non-trivial, as outcomes vary depending on factors such as workloads, message sizes, and hardware configurations. For instance, our experiments reveal that for the same CPU allocation, network band width can vary between 0.4× and 3.5×. In this paper, we introduce TASADOR, a machine learning (ML) framework for network bandwidth to CPU translations. TASADOR predicts “appropriate” CPU budgets for new workloads to meet their bandwidth requirements. We demonstrate that TASADOR completes data collection and training in ∼16 minutes while achieving target bandwidth accurately with ∼6.5× less CPU utilization compared to existing schemes. Our experiments also confirm that TASADOR is effective across diverse workloads, message sizes, and hardware configurations.

GPU sharing aims to enhance the efficiency of GPU utilization by running distributed deep learning training jobs concurrently. However, GPU sharing poses a significant challenge: the increase in job completion time (JCT) caused by interference between jobs is inconsistent, complicating job scheduling. Our experiments reveal that the degree of JCT increase varies by as much as-3.7x. While previous studies have analyzed this JCT inconsistency problem, none of them have been able to minimize the inconsistency. We propose TensorShare, a proactive GPU sharing technique that leverages a deep learning model to predict the extent of JCT increase. This study defines a new metric, called GPU SLA, which represents the upper threshold of JCT increase. TensorShare then introduces a novel scheduler that proactively identifies which jobs meet GPU SLA while minimizing the JCT increase. Our evaluation shows that TensorShare improves GPU SLA satisfaction rates by 26.1x-47.3x and reduces the JCT increase by 37%-60%. Furthermore, we evaluate TensorShare with large language models that are not included in training TensorShare's prediction model, achieving-7x and-10.3x improvements in GPU SLA satisfaction and JCT inconsistency, respectively.

Key-value (KV) caching for shared prefixes is essential for high-throughput large language model (LLM) serving, but it faces critical challenges in emerging diffusion language models (DLMs). In DLMs, bidirectional attention means that updating any token dynamically alters the entire context and its corresponding KVs. Thus, existing caching techniques developed for LLMs, which assume that KVs remain invariant once computed, corrupt the shared prefix KVs. Our experiments show that applying these techniques to DLMs causes model accuracy to collapse to near zero. To unlock high-throughput DLM serving, we propose bidirectional prefix caching, bicache, the first KV caching technique for shared prefixes in DLMs. bicache is designed based on key observations from our comprehensive analysis: shared prefix KVs remain stable and reusable in shallow layers, while the depth of shallow layers depends on the fraction of shared prefix tokens in each request. Thus, bicache dynamically identifies a safe layer depth for reusing shared prefix KVs and eliminates redundant computation. Evaluations demonstrate that bicache significantly improves serving throughput by 36.3

This study presents a cloud-based platform for managing personal digital biomarker data collected from wearable and mobile devices. The platform aims to address four essential challenges for personalized healthcare: ensuring data integrity, supporting various biomarker data types, maintaining measurement reliability, and enabling system scalability. To protect data integrity and privacy, the platform uses private blockchain technology. It also supports heterogeneous data formats, such as time-series ECG signals and dietary images, through standardized interfaces. Measurement reliability is ensured by Gaussian mixture model that successfully detect faulty data with 100% recall. In real-world clinical tests with patients, the platform detects all attempts of data tampering and processes diverse biomarker data with an average latency of less than 1.76 s. The platform’s scalability maintains stable performance under high data loads, and practical use is demonstrated through integration with our mobile application and biomarker web server. We validate the platform by measuring actual biomarker data from real-world atrial fibrillation patients and demonstrate its capability to securely and efficiently integrate wearable biomarker data for personalized healthcare.

Selecting a parallelism strategy - the configuration of data, tensor, and pipeline parallelism degrees together with micro- and global-batch sizes - largely determines the training efficiency of large language models. State-of-the-art methods search for a parallelism strategy offline and select the single strategy that minimizes per-iteration time. But we find that they neglect the target validation perplexity and time-to-perplexity (TTP). In particular, our analysis reveals that the best strategy yielding the fastest perplexity improvement changes multiple times during training. As a result, state-of-the-art methods are 1.8-11.4x slower in TTP than the strategy sequence that selects the best strategy at each iteration. This paper proposes CONA, a new training method that introduces online strategy chaining. Instead of a single strategy selected offline, CONA ranks candidate strategies during training using a surrogate metric built from compute throughput and gradient statistics, and switches the current strategy to a new strategy with a higher metric. In our evaluation with GPT-3 1.3B, BERT-Large, and Llama-3.2-1B, CONA reaches the target validation perplexity 1.4-9.6x faster than state-of-the-art methods. Moreover, CONA closely tracks the perplexity achieved by the sequence that selects the best strategy at each iteration, within 2.6

Long-context LLM serving requires offloading KV caches to host-memory and SSDs, but existing mechanisms are not designed for such long contexts. We observe significant inefficiencies in current KV caching in long contexts: high serving latency ~30.7 s, exceeding the typical TTFT requirement of 10 s by more than 3x. Our in-depth analysis explains two major reasons: (1) retrieval is serialized through host-memory and SSD, leaving other host-memory modules and SSDs underutilized, and (2) SSD-based KV retrieval spends 84% of its time in the kernel filesystem rather than actual device access. To address the problems, we propose unified KV pooling, which aggregates multiple host-memory modules and SSDs into a single logical pool and distributes KV caches across devices based on their bandwidth. To eliminate the filesystem overhead, we design KV-passthrough, which bypasses the kernel filesystem and directly accesses SSD-resident KV caches from user space via SPDK. Across evaluations on LLaMA 3.1-8B, GPT-OSS-20B, and Qwen3-30B-A3B, unified KV pooling reduces TTFT in long-contexts ~4.1x over state-of-the-art techniques, all making under 10 s. It also reduces blocked I/O time by up to 23.2x by eliminating filesystem overhead.

Python 3.13/3.14 introduces a new copy-and-patch JIT that reuses stencil templates. Despite its performance benefits, we find that it enlarges the Python attack surface. Specifically,we perform a systematic security analysis of this stencil-based architecture and uncover a new class of attack vectors, which we term StencilLeak. Unlike prior JIT exploits that rely on attacker-injected data or dynamic recompilation, StencilLeak exploits the inherent determinism of legitimate stencil templates. In a standard Python environment with active defenses (ASLR, AppArmor) enabled, we show that our pure Python code with user privilege can successfully execute arbitrary binary code on x86_64. Across 500 attack attempts, the StencilLeak attack succeeds in 50.2% of single attempts, reaches 99.2% within 7 attempts, and completes as fast as 4.6 seconds. These results demonstrate that although CPython’s JIT is now an experimental feature, the structural predictability of copy-and-patch architectures warrants rigorous security evaluation.

Datacenter network topology contains multiple paths between server machines, with each path assigned a weight. Software switches perform traffic splitting, an essential networking operation in datacenters. Previous studies leveraged software switches to distribute network connections across paths, under the assumption that the software switches accurately divide connections according to path weights. However, our experiments reveal that current traffic splitting techniques exhibit significant inaccuracy and resource inefficiency. Consequently, real-world datacenter services (e.g., data mining and deep learning) experience communication completion times that are ∼2.7× longer than the ideal. To address these problems, we propose VALO, a new traffic splitting technique for software switches, to accomplish two goals: high accuracy and resource-efficiency. For the goals, we introduce new concepts: score graph and VALO gravity. We implement VALO using the de-facto software switch, Open vSwitch, and evaluate it thoroughly. On average, VALO achieves 13.1× better accuracy and 25.4× better resource efficiency compared to existing techniques, with maximum improvements reaching up to 34.8× and 67.7×, respectively. As a result, VALO demonstrates 1.3×-2.5× faster average communication completion times for real-world datacenter services compared to existing techniques.

As deep learning models in agentic AI systems grow in scale and complexity, GPU memory requirements increase and often exceed the available GPU memory capacity, so that out-of-memory (OoM) errors occur. It is well known that OoM interrupts the whole training itself and wastes substantial computational resources. Therefore, to prevent OoM, accurate prediction of GPU memory usage is essential. However, previous studies focus only on unimodal architectures and fail to generalize to multimodal models, even though the multimodal models are a common choice in agentic AI systems. To address this limitation, we propose a framework that predicts the peak GPU memory usage by analyzing the model architecture and training behavior of multimodal models. Specifically, the framework decomposes the multimodal model into its constituent layers and applies factorization to estimate the memory usage of each layer. Our evaluation shows that our framework achieves high prediction accuracy of 8.7

This article explores the computing and communication overhead of network processing in Internet of Things (IoT) devices, focusing on containers, a major building block for the edge computing. Our experiments reveal that containers on IoT devices suffer similar to 2.6 x higher CPU usage for SoftIRQ processing, similar to 59% less network throughput, and 2x higher per-packet latency on average than native processes. While several existing studies enhance networking performance, they often sacrifice interoperability by requiring special hardware or modifying networking semantics or APIs. Thus, we design and implement a kernel networking accelerator, called SCON, that maintains interoperability, crucial for IoT devices. SCON addresses major bottlenecks in container networking through system-level profiling. We evaluate SCON with three types of IoT devices. On the Raspberry Pi 4, SCON reduces the latencies of major IoT application protocols (e.g., HTTP and MQTT) by similar to 10 x , achieving a similar level of latency to the native process. Further analysis shows that SCON reduces CPU usage for SoftIRQ processing by similar to 26%. We also report similar improvements on the other two IoT devices. Our conclusion is that SCON is unique in significantly reducing the computing and communication overhead of container networking in IoT devices while maintaining interoperability. Furthermore, it works consistently across different types of devices, whether wired or wireless, and regardless of heavy or sporadic traffic.

Blockchain is increasingly offered as blockchain-as-a-service (BaaS) by cloud service providers. However, configuring BaaS appropriately for optimal performance and reliability resorts to try-and-error. A key challenge is that BaaS is often perceived as a “black-box,” leading to uncertainties in performance and resource provisioning. Previous studies attempted to address this challenge; however, the impacts of both vertical and horizontal scaling remain elusive. To this end, we present machine learning-based models to predict network reliability and throughput based on scaling configurations. In our evaluation, the models exhibit prediction errors of ∼1.9%, which is highly accurate and can be applied in the real-world.

Federated learning (FL) is an appealing model training technique that utilizes heterogeneous datasets and user devices, ensuring user data privacy. Existing FL research proposed device selection schemes to balance the computing speeds of devices. However, we observe that these schemes compromise prediction accuracy by similar to 57.7%. To solve this problem, we present Harmonia that enhances prediction accuracy, while also balancing the diverse computing speeds of devices. Our evaluation shows that Harmonia improves prediction accuracy by similar to 1.7x over existing schemes.

Blockchain-as-a-service (BaaS) in cloud datacenters is gaining widespread attention due to its high performance and privacy. However, existing BaaS solutions lack a method for deciding the proper placement of blockchain nodes across virtual machines in worldwide datacenters to achieve desired performance. Our motivating experiments show that transaction processing performance (TPS) varies similar to 31.6% depending on the placements. To provide an automatic placement solution for BaaS, we propose Cyan that predicts the TPS for blockchain node placements. Our evaluations on Google Cloud Platform demonstrate that Cyan improves the TPS guarantee similar to 2.39x compared to existing techniques.

This article investigates how to satisfy network performance requirements that are crucial in achieving the service level objectives (SLOs) in clouds. Traditional techniques for network performance management have a limited ability to satisfy the network SLOs. Our in-depth analysis reveals that the fundamental reason comes from decoupling of the CPU scheduler and the network traffic controller as the current CPU scheduler is not aware of such network requirements but only provides a fair-share amount of CPU to all containers. Thus, the container cannot perform the amount of network processing as needed to satisfy its SLO when the CPU allocation is insufficient. In this article, we propose Autothrottle that dynamically adjusts the CPU allocation for the containers to satisfy their network SLOs. The key element of Autothrottle is a throttle algorithm that autonomously determines the amount of CPU for each container needed to satisfy the requirement. We implement Autothrottle in the Linux kernel and evaluate it with massive real-world workloads such as Apache Kafka. Our evaluation results show that Autothrottle successfully satisfies the given network SLO only with a 2% gap while the existing scheme achieves 20% less than the SLO. We further observe that Autothrottle also reduces the CPU overhead in network processing by 19%, improving the network throughput by 27% compared to the existing scheme.

We introduce TeaVisor that provides bandwidth isolation guarantee for network virtualization (NV) based on software-defined networking (SDN). SDN-based NV (SDN-NV) offers many benefits to clouds, such as topology and address virtualization while allowing flexible resource provisioning, control, and monitoring on virtual networks. In SDN-NV, however, routing is done by tenants independently; thus, existing studies have difficulties in bandwidth isolation guarantee due to the overloaded link problem. Bandwidth isolation guarantee is essential for providing stable and reliable throughput on network services in SDN-NV. Without bandwidth isolation guarantee, tenants suffer degraded service qualities and significant loss in revenue. To address this problem, we design and implement TeaVisor in three components: path virtualization, bandwidth reservation, and path establishment. Through extensive experiments, TeaVisor shows that bandwidth isolation is guaranteed with near-zero errors, which is three orders of magnitude better than existing studies. In addition, TeaVisor guarantees the minimum and maximum bandwidth at the same time. We also present an overhead analysis of TeaVisor in control traffic and memory consumption.

This article presents Elixir, an automated prediction model formulation framework for control traffic using machine learning. Control traffic is vital in software-defined networking (SDN) systems because it determines the reliability and scalability of the entire system. Various studies have sought to design control traffic prediction models for the proper provisioning and planning of SDN systems. However, previously proposed models are based on descriptive modeling, well-suited for only specific SDN system instances. Furthermore, these models exhibit poor accuracy (errors of up to 85%) because of the heterogeneity of SDN systems. Because descriptive modeling requires a significant amount of human contemplation, it is impossible to formulate adequate prediction models for countless SDN system instances. Elixir addresses this problem by applying machine learning. Elixir starts the model formulation through self-generated datasets. Then, Elixir searches prediction models to fit the accuracy for respective SDN systems. Also, Elixir picks robust models that exhibit reasonable accuracy even in a network topology that differs from the topology used for model training. We evaluate the Elixir framework on nine heterogeneous SDN systems. As a key outcome, Elixir significantly reduces prediction errors, achieving up to 10.6x improvement compared to the previous model for control traffic throughput of OpenDayLight controller.