This paper investigates the online scheduling problem of path selection and workload allocation for in-network computing. We formulate the problem as a mixed-integer programming model that jointly minimizes server operating cost and end-to-end latency, and prove that it is NP-hard. To enable efficient online decision-making, we further transform the problem into a Markov decision process. Based on this formulation, we propose a reinforcement learning–based scheduling algorithm using proximal policy optimization (PPO), which incorporates a Long Short-Term Memory (LSTM)-based temporal encoder to capture historical deployment information and dynamically evolving network states. This design allows the algorithm to make adaptive, queueing-aware scheduling decisions in dynamic network environments. Extensive simulation results demonstrate that the proposed algorithm is robust and consistently outperforms state-of-the-art baseline methods in terms of the joint cost–latency performance metric.

As the foundational component of versatile AI applications, training an multimodal large language model (MLLM) relies on multimodal datasets with dynamic modality mixture proportions and sample length distributions. However, existing MLLM systems remain inefficient under dynamic workloads, due to statically coupled decisions of resource allocation and model parallelization between encoders and the LLM backbone. This paper presents MegaScale-Omni, an industrial-grade MLLM training system tailored for dynamic workload adaption and hyper-scale deployment. MegaScale-Omni is built upon the training scheme of encoder-LLM multiplexing with three key innovations: (1) Decoupled parallelism strategies with long-short sequence parallelism for encoders to process variable-length samples, and full-fledged 5D parallelism for the LLM backbone, both organized under a communication-efficient parallelization layout. (2) Unified encoder-LLM representations for flexible, extensible colocation, and a new paradigm of encoder-LLM joint pipeline with workload resilience. (3) Workload balancing techniques via decentralized grouped reordering in data loaders and adaptive resharding from encoder to LLM ranks. MegaScale-Omni is deployed as the foundation of our in-house large-scale MLLM training tasks with thousands of GPUs. Our experimental results demonstrate 1.27×-7.57× throughput improvement under production-grade dynamic workloads, as compared to four state-of-the-art systems.

The Internet of Things (IoT) is gathering paces in the new era of Industry 4.0, and the Digital Twin (DT) technology bridges the gap between the bursting amounts of data generated by IoT devices and the user requirements for real-time data processing. DT services maintain living digital models of physical objects, and a DT network enables comprehensive service provisioning with the global knowledge of a group of DTs. On the other hand, exposing serverless computing at network edges, the recent advances in Mobile Edge Computing (MEC) introduce new inspirations to the DT landscape that ensure fine-grained resource management and low network-wide delay of DT services. However, social relationships among IoT devices and DT data privacy impact DT orchestrations. In this paper, we first design a differential privacy-based federated learning framework to build a DT network for DT services in response to user requests in an MEC, thereby enhancing the Quality of Services (QoS). Built upon the proposed framework, we then formulate two novel social-aware DT placement problems: the static social-aware S_DT placement problem, and the dynamic social-aware S_DT placement problem, respectively. We also show the NP-hardness of the defined problems. Then, we formulate an Integer Linear Program (ILP) solution to the static social-aware S_DT placement problem when the problem size is small; otherwise we develop an approximation algorithm with a provable approximation ratio for it. Third, we study the dynamic social-aware S_DT placement problem when requests arrive one by one without the knowledge of future request arrivals over the time horizon, for which we devise an online algorithm with a provable competitive ratio. Finally, we conduct simulations to evaluate the performance of the proposed algorithms. Simulation results show that the proposed algorithms outperform their counterparts, improving the performance compared with their baselines by no less than 14.9%.

Recently, millimeter-wave communication has attracted extensive research interests for its advantages of large bandwidth and low latency. Meanwhile, Age of Information (AoI) has been proposed as a key performance indicator reflecting the real-time performance from the point of destination. However, in millimeter-wave networks, there are still few studies on multicast scheduling mechanisms targeting AoI. Therefore, this paper mainly studies the problem of optimizing AoI in millimeter-wave networks, aiming to minimize the Broadcast Age of Information (BAoI). Firstly, this paper constructs a system model including multi-base stations, multi-user devices and multicast groups, and formulates the problem optimizing BAoI with beam selection and collaborative scheduling. For the problem of signal deafness, an efficient beam scheduling and resource allocation strategy is proposed to avoid data transmission delay caused by UE's limited single connection capability. Due to the high-dimensional nonlinear characteristics of beam parameter optimization, a multicast scheduling algorithm based on improved Differential Evolution is proposed, which integrates adaptive mutation strategies to enhance the global search ability and convergence efficiency in the high-dimensional nonlinear search space. The simulation experiments show that the proposed algorithm can significantly reduce BAoI under different system parameter settings, verifying its effectiveness in optimizing BAoI.

As the context window of Large Language Models expands to accommodate increasingly complex tasks, the KV cache emerges as a performance bottleneck. While existing compression methods reduce the memory footprint and access, they either suffer severe accuracy degradation, or incur substantial global selection overhead that scales linearly with context length. To address this dilemma, we propose LocalKV, a sparse decoding framework for efficient long-context LLM inference that preserves accuracy, leveraging algorithm-system co-optimization. On the algorithm side, LocalKV leverages the locality of sparse attention to identify important tokens only from the neighbors of previously selected tokens, avoiding costly global selection. On the system side, LocalKV eliminates the local tracking overhead by fusing it into the sparse Flash Attention kernel, and utilize asynchronous CPU-to-GPU prefetching to completely overlap the KV transfer latency. Extensive experiments show that LocalKV maintains near-lossless generation quality, while delivering a 1.6 × decoding speedup over state-of-the-art methods.

With the rapid development of AI and deep learning, computational demands are increasing significantly. While GPUs excel in parallel computing, they fall short in terms of energy efficiency, specialization, and processing latency. In contrast, Neural Processing Units (NPUs), such as the Ascend NPUs, designed specifically for deep learning tasks, demonstrate superior performance. However, the architecture specialization makes operator development more challenging, leading to a reliance on manual tuning and optimization, which incurs significant time cost and developing effort. To address this issue, we propose NPUMeter, an automatic operator optimization framework for Ascend NPUs built upon accurate and comprehensive analytical performance models. NPUMeter comprises two components: (1) an analytical performance model that accurately estimates operator latency on NPU given different configurations of optimization parameters; (2) an efficient design space exploration (DSE) algorithm that automatically searches for the optimal parameter configuration in a large design space within minutes. Experimental results demonstrate that NPUMeter achieves high estimation accuracy, with an average error below 5%. It effectively generates near-optimal configurations for various operators, achieving up to a 1.46× performance speedup compared to the configuration generated by the Ascend C compiler while reducing the DSE time from hours to minutes.

Artificial intelligence (AI) has been increasingly applied to gaming, with large language models (LLMs) playing a key role in character control. However, efficiently co-locating game rendering and LLM inference on one GPU presents challenges due to resource constraints, diverse latency requirements, and fine-grained task scheduling. We propose LEGO, an algorithm-system co-design that enables the efficient co-location of LLM inference and game rendering tasks. Algorithmwise, LEGO features a resource-oriented layer-skipping adaptor, which distills knowledge from skipped layers to reduce computational demand while maintaining inference accuracy. System-wise, LEGO proposes a headroom-maximizing LLM scheduler, which dynamically partitions inference tasks to utilize available rendering headroom. Evaluations on an Nvidia RTX 4090 show that LEGO meets latency targets in all scenarios, improves rendering headroom utilization by up to 28.6 %, and reduces LLM inference accuracy loss by up to 86.3 % compared to current layer-skipping approaches.

With the wide adoption of Multimodal Models (MMs) in real-world scenarios, it is significant to efficiently train emerging MMs that exhibit increasingly complex module architectures. For MM deployment, existing works allocate a GPU to only one MM module in a temporal-multiplexing manner; this compromises training efficiency because a single module often fails to achieve high GPU utilization. To improve GPU utilization and enable efficient MM training, we propose deploying MMs in a temporal-spatial multiplexing manner, allowing multiple MM modules to colocate on a GPU with well-controlled resource quotas. In this paper, we propose Apollo, an efficient MM training system that applies temporal-spatial multiplexing. We first develop a flexible and lightweight execution engine that supports MM training with arbitrary resource quotas, and then build a comprehensive and accurate performance model to estimate module execution time under different allocation plans. With the performance model, we further adopt effective heuristics to derive high-quality MM deployment plans efficiently. Testbed experiments confirm that Apollo effectively improves the training efficiency of popular MMs, with a training speedup of up to 1.31x.

As the inter-container data transmission time constitutes a large portion of the overall execution time in contemporary containerized applications, it is natural to group tightly coupled containers into pods to lower the transmission time. However, this container composition leads to longer application waiting time, as all containers within a pod must be started before the pod can provide services. In this paper, we investigate how to group the containers into pods and allocate these pods onto servers with heterogeneous resource capacities and data transmission time, aiming to achieve the minimum completion time. The problem is formulated in quadratic mixed integer programming (QMIP) form and is proven to be NP-hard. Based on the QMIP formulation, we then propose a randomized rounding based multiple containers in one pod (MCOP) placement strategy with reduced computational complexity. Trace-driven experimental results show that MCOP reduces the application completion time by 27.26% to 55.81% compared to state-of-the-art solutions.

Federated Learning (FL) is a distributed paradigm that supports collaborated model training while preserving data privacy, where clients periodically synchronize their local gradients once after multiple local iterations. Due to non-uniform data distribution and poor network condition, FL processes often suffer degraded training accuracy and efficiency. In this work, we analyze the microscopic parameter variation behaviors in FL, and find that an effective method to improve FL accuracy is to switch to more frequent synchronization at proper moments. In particular, such frequency-tuning moments—which can be detected from gradient characteristics—are heterogeneous across different parameters. Motivated by such observations, we propose Parameter-Adaptive Synchronization (PAS), a FL scheme that adaptively tunes the synchronization period for each scalar parameter. The benefits of PAS are two-fold: By switching to more frequent synchronization when necessary, we can improve the FL training accuracy; by synchronizing different parameters independently, we can enable communication-computation overlapping and enhance the network utilization. We have theoretically demonstrated the convergence validity of PAS, and have further extended it with adaptive sparsification capability to jointly reduce the overall communication volume. We implemented PAS atop PyTorch, and extensive experiments show that it can substantially improve FL performance in both accuracy and communication efficiency.

Federated Learning (FL) allows edge clients to collaborate in model training with data privacy preserved, yet it is known to suffer low training efficiency and model accuracy. Given that efficiency and accuracy are usually conflicting objectives, existing practices increasingly employ an adaptive scheme that changes the FL configurations (e.g., quantization or sparsification level) based on runtime training status, for which accurate training diagnosis—used for guiding the optimization actions—is crucial. However, while training diagnosis is a common task shared by different optimization schemes, existing works propose their diagnosis methods in an ad-hoc manner, which yield multiple limitations. First, the diagnosis metric in an optimization scheme may sometimes be less accurate than others; second, existing schemes fail to fully exploit the diagnosis result by applying it for only one optimization action; third, existing methods usually do not perceive cross-client data heterogeneity, failing to simultaneously enhance FL accuracy. To tackle those limitations, we make a systematical study on the training diagnosis methods of multiple optimization schemes, and propose metric grafting—replacing a scheme's diagnosis metric with a better one to improve the training performance. Moreover, to fully exploit the potential of training diagnosis, we build a system platform that supports flexible combinations of training diagnosis and optimization actions (i.e., single-diagnosis-multiple actions and multiple-diagnosis-multiple-actions). Evaluation on testbeds show that, with metric grafting and advanced diagnosis action combinations, we can substantially improve the efficiency and accuracy performance of FL.

Federated Learning (FL) enables collaborate model training without privacy violation, where clients periodically report their updates to the server in communication rounds. Due to heterogeneous resource and limited bandwidth, FL processes often suffer from low efficiency. Existing works in that regard are oblivious to the intra-round execution status on clients, failing to tackle runtime stragglers or hide the communication overheads for some early-converged layers. In this paper, we propose FedCA, a novel mechanism that allows clients to autonomously exploit intra-round training status for higher efficiency while preserving accuracy performance. We first devise a metric to help quantify the statistical contribution of different iterations in a round, which can be efficiently profiled at runtime with the periodical sampling strategy. With the instantaneous system and statistical status, to improve computation efficiency, clients under FedCA can adaptively determine the intra-round workloads based on a utility function depicting the marginal computation benefit. Besides, to mitigate the communication bottleneck, for some parameters attaining fast local convergence, clients under FedCA can eagerly transmit their updates to the FL server prior to round completion. We also extend FedCA to FedCA+, integrating speculative sparsification to futher reduce the cumulative communication amount. We implemented FedCA and FedCA+ atop PyTorch, and large-scale experiments show that they can improve the FL efficiency by up to 45.3%.

Graph processing workloads are increasingly being migrated to the cloud. With the growing adoption of serverless computing, graph processing gains advantages such as cost-effectiveness and resource elasticity. However, existing graph processing systems with monolithic function architecture struggle to detect intra-job resource elasticity and suffer from significant communication overhead. In this paper, we present FaaSBoard, a graph processing system with a disaggregated serverless architecture powered entirely by serverless cloud services. FaaSBoard features a multi-tier data communication mechanism and an autonomous-elastic computing mechanism. Specifically, these two mechanisms are realized through image-based graph loading for faster starts, proxy-based collective communication leveraging high-bandwidth shared memory, 2D balanced graph partitioning for improved load balance, and a proactive terminate-and-respawn mechanism enabling fine-grained elasticity. Together, these four techniques collectively enhance both resource and overall execution efficiency. Experimental results demonstrate that FaaSBoard delivers up to 3.8× higher compute performance and reduces monetary cost by up to 61.5% compared to FaaSGraph, the current state-of-the-art serverless-based graph processing system. The source code of FaaSBoard is publicly available athttps://github.com/SJTU-Liquid/FaaSBoard.

As large language models scale to longer contexts, loading the growing KV cache during attention computation becomes a critical bottleneck. Previous work has shown that attention computation is dominated by a small subset of tokens. This motivates block sparse attention methods that partition the KV cache into fixed-size blocks and selectively compute attention over those blocks exhibiting high importance. However, these methods assign a uniform block size across all attention heads, implicitly assuming homogeneous behavior throughout the model. Our analysis reveals that this assumption is flawed: attention heads exhibit widely varying sensitivity to block granularity, and uniformity leads to suboptimal accuracy. We present AB-Sparse, a training-free algorithm-system co-designed framework that improves accuracy while preserving throughput. AB-Sparse introduces lightweight adaptive block size allocation across attention heads to improve accuracy. To compensate for the additional memory overhead, it further employs lossless block centroid quantization. In addition, custom GPU kernels are developed to support efficient execution with variable block sizes. Evaluation results demonstrate that AB-Sparse achieves an accuracy improvement of up to 5.43

In Unmanned Aerial Vehicle-assisted Mobile Edge Computing (UAV-MEC), dynamic workloads and limited onboard energy pose significant challenges for efficient task scheduling and long-term mission sustainability. Cognitively-inspired computing paradigms provide an intelligent solution by enabling UAVs to perceive environments, learn from experience, and make adaptive decisions. This paper proposes a TS-Diff (Two-Stage Diffusion Policy) framework for joint task offloading, trajectory planning, and energy harvesting. A brief Soft Actor-Critic pre-training stage first constructs an exploratory experience memory buffer to address the cold-start issue of diffusion models. A Diffusion Policy Actor is then employed to iteratively generate robust continuous control actions, forming a perception–decision–action loop for adaptive UAV control. Experimental results show that TS-Diff achieves a final average return of approximately -145, improving performance by about 20

As uncrewed aerial vehicles (UAV) can provide edge service flexibility, the UAV-assisted edge computing (UAEC) has attracted much attention recently. Computation offloading and service function chain (SFC) deployment are closely related and interactional, which can influence the efficient usage of limited resources in UAEC. Therefore, a joint computation offloading and SFC deployment (JCOSD) scheme is researched in this paper. The offloaded tasks will be processed by the ordered virtual network functions (VNF) in SFC, and the VNF can be deployed on UAV. The delay and energy consumption are minimized at the same time. The constraints on task requests and computing capabilities of heterogeneous nodes are considered. Considering the complex network features and the relationship between heterogeneous nodes in UAEC, a graph neural network (GNN)-based algorithm is proposed to extract network node features based on local observations. The edge weights including bandwidth, expected delay, and expected energy consumption are applied to effectively reflect the node importance. Then the global feature embeddings from GNN are used as the state input of a multi-head double deep Q network. Computation offloading and SFC deployment strategies are jointly determined. The numerical results have shown the effectiveness of the proposed JCOSD scheme in different cases, and JCOSD can achieve up to 22.51%, 47.10%, 56.69%, and 42.29% better performance than other four benchmarks.

In this article, we present the joint optimization of computation path selection and workload allocation for directed acyclic graph (DAG) tasks in edge computing networks. Existing works primarily focus on end-to-end latency and are often restricted to simple task chains, neglecting critical factors such as server operational costs and the dynamics of the arrival of tasks. To bridge this gap, we formulate the online scheduling problem as a mixed integer program to minimize server operational costs and latency. We then decompose this problem into a minimum-latency path selection subproblem and a task scheduling subproblem formulated as a Markov decision process (MDP). Our solution consists of a latency-aware transmission scheduling (LATS) algorithm and a novel online scheduler based on proximal policy optimization (PPO). Furthermore, we leverage graph neural networks (GNNs) and long short-term memory (LSTM) networks to encode the system state, thereby significantly improving the agent's perception of the complex environment. Finally, extensive simulation results demonstrate that the proposed algorithm shows good adaptability and outperforms the state-of-the-art algorithms.

Microservices are widely shared in production user-facing applications. These shared microservices have various resource usage patterns when queries from different call graphs of different services access them. However, existing microservice management works fail to efficiently scale resources for them, mainly due to the lack of fine-grained scheduling of diverse queries. We therefore propose Delphinus , a runtime system that efficiently manages resources for shared microservices while ensuring the Quality-of-Service (QoS). Delphinus comprises a group-oriented query scheduler and a borrowing-based load adapter . The query scheduler identifies diverse queries, groups the containers of shared microservices, and schedules the queries into separate groups. The load adapter efficiently scales resources for shared microservices, and fully utilizes the idle containers among groups when the loads of diverse queries change. Results show that Delphinus reduces CPU and memory usage by 40.1% and 36.4% for shared microservices, respectively, compared to state-of-the-art works.