To address the increasing computational demands driven by the development of large-scale pretrained AI models, hybrid-cloud environments have emerged as an effective paradigm for elastically extending computational resources while balancing the cloud cost and job completion time (JCT). Recent advanced schedulers aim to dynamically assign waiting budgets according to job-specific computational loads. However, these schedulers often rely on deterministic, single-shot mathematical models that cannot be adequately adapted to real-time fluctuations in the cluster state. Under bursty workloads, the absence of multi-round, feedback-driven load awareness can allow long-tail jobs to monopolize scarce resources, thereby increasing tail latency and cloud costs. To address this issue, we propose LHRSLTW, a load-aware hybrid-cloud resource scheduling framework for large-scale training workloads. The key innovations of our framework are as follows: $i$) Multi-round cardinality-aware resource profiling algorithms (MCRPs) are established, which introduce cardinal queries into hybrid-cloud scheduling. Through a three-phase iterative process of load sensitivity determination, starvation detection, and trend analysis, this mechanism dynamically generates resource-matching factors and significantly increases job-node affinity and scheduling flexibility under bursty workloads. $ii$) Load-aware adaptive deep reinforcement learning for dynamic workloads (LAADRL-DW) is developed. By introducing dynamic learning factors, online normalization, and asynchronous multithreaded parallel updates, LAADRL-DW achieves real-time adaptive optimization of scheduling strategies and responds effectively to load fluctuations and heterogeneous resource contention. $iii$) An intelligent offloading system (IOS) is designed. This module relies on a hierarchical offloading strategy that accounts for the queue length, resource saturation, hardware affinity, and cost efficiency. By implementing dynamic threshold triggering, affinity-based scoring, overload prediction, and cost-efficiency balancing, the IOS ensures local execution of high-priority and large-scale tasks while flexibly offloading inefficient or CPU-intensive tasks to the cloud. The experimental results demonstrate that, under moderate to low workload conditions, LHRS-LTW achieves superior cost savings, providing additional savings of up to 40.56%. Under highly bursty workload scenarios, the collaborative interaction among the proposed modules achieves cost savings of up to 17% and increases cluster utilization to 99.2%.

As Latency-critical (LC) tasks often experience diurnal load patterns, co-locating them with best-effort (BE) tasks improves resource utilization. Prior work allocates entire CPU cores between co-located tasks, due to the incapability of handling the interference with CPU sharing. We observed that the root cause of the interference on the same core is the inherent wake-up delay in the operating system scheduler, the wait time that a process can obtain the CPU cycles after it is woken up. Based on the finding, we propose WDP, a scheme that efficiently improves the throughput of BE tasks while ensuring QoS, leveraging CPU sharing. WDP comprises a wake-up delay-driven preemption mechanism and a preemption-based CPU manager. The preemption mechanism enables controlled preemption to reduce the wake-up delay of LC tasks with adjustable preemption capacity. Adopting the novel preemption mechanism, the CPU manager allocates CPU resources in a fine-grained manner among co-located tasks. Compared with the representative prior method, WDP improves the throughput of BE tasks by 31.2% on average while ensuring the QoS of co-located LC tasks.

Edge device traffic prediction is crucial for autonomous network control and management. However, the rapid proliferation of smart 5G networks results in increasingly heterogeneous, dynamic, and complex traffic loads on edge nodes, rendering traditional short-term prediction methods insufficient for medium- and long-term network resource scheduling. To address this, we propose a novel multi-node lightweight long-sequence deep learning-based prediction architecture (EMC-LSP) to effectively capture complex long- and short-term correlations in edge environments. Specifically, EMC-LSP employs frequency-domain hard-attention decomposition to separately model non-stationary high-frequency and low-frequency traffic, utilizes two-layer null frequency-domain convolution for long-term low-frequency similarity, and designs a high-frequency interpolation-based prediction method. In extensive tests on 18 datasets, EMC-LSP demonstrated superior performance, reducing the average prediction error of MSE and MAE by 15.20% while decreasing model parameters by 50 times.

Many applications adopt multithreading to increase the concurrency or computational efficiency. In this scenario, the active threads are often much more than the cores of a server, and are scheduled with the operating system. However, we observe that the current load balancing mechanism based on scheduling domains leads to poor performance on asymmetric NUMA machines. Our investigation shows that the poor performance is due to the unnecessary “far” scheduling with high cost, because the current algorithm of building scheduling domains neglects the physical relevance of NUMA nodes in a machine. We therefore propose a physical relevance-based algorithm to construct scheduling domains on asymmetric NUMA machines to reduce the scheduling cost. Experimental results show that the proposed scheduling domains improve the real-world applications by 14.94

Tenants usually rent virtual machines (VMs) to deploy their applications, while those VMs are generally colocated on the same physical machine by the cloud provider to improve resource utilization. Meanwhile, modern CPUs are evolving towards heterogeneous multicore architecture, which integrates powerful brawny cores and power-efficient wimpy cores into a CPU of a machine. This paper focuses on how to allocate CPU resources when co-locating VMs on the heterogeneous architecture. When a VM is created by the Kernel-based Virtual Machine (KVM) virtualization technology, it consists of vCPU threads and Emulator threads. Our investigation shows that properly setting the range of CPU cores where Emulator threads are allowed to run can further improve the VM performance, when using Linux Completely Fair Scheduler (CFS) to schedule VMs. However, it is challenging to find the proper range setting because running different applications in the VM leads to nonidentical optimal range settings, and the VM is usually a black-box from the perspective of the cloud provider. We therefore propose Themis, a runtime system comprising a core scheduler and a load monitor. The core scheduler finds the proper range setting based on low-level information, while the load monitor identifies whether the load of the VM is significantly changed and the range is required to be adjusted. The experimental results show that Themis improves the performance of popular cloud applications by up to 48.4% compared with prior work.

Latency-critical (LC) services are usually co-located with best-effort applications to improve the resource utilization. Lots of studies have been proposed to guarantee the performance of LC services in the co-location by managing shared resources. However, even the LC service has enough resources, its performance may still severely degrade because of memory reclaim caused by the operating system. We therefore propose Priority Memory Reclaim (PMR) which can eliminate impact of memory reclaim on LC services as much as possible. PMR consists of two techniques: priority page swapping and adaptive watermark configuration. Experiment results show that PMR can greatly improve performance of LC services. Specifically, PMR can reduce the 99%-ile latency and maximum latency of LC services by up to 92.32% and 95.87% while improving the throughput by up to 4.75x.

Latency-critical applications directly interact with end users and often experience the diurnal load pattern. In production, best-effort applications are often co-located with them to utilize the idle cores at the low load. Meanwhile, modern computers are evolving towards heterogeneous NUMA architecture, where the cores have different computation abilities, memory access latencies and network communication delays. Prior co-location scheduling work did not consider the NUMA architecture, and failed to maximize the throughput of best-effort applications while ensuring the required QoS of latency-critical applications. Our investigation shows that NUMA effect has complex impacts on the latency of latency-critical applications and the throughput of best-effort applications. We therefore propose PAC, a preference-aware co-location scheduling scheme that considers the NUMA effect for heterogeneous NUMA architectures. PAC has a performance monitor and a core scheduler. Specifically, the performance monitor identifies the "dangerous" latency-critical applications that require upgrading core allocations. We propose two low-overhead scheduling strategies for the scheduler. The strategies identify the bottlenecks of applications and adjust core allocations accordingly. Experimental result shows that PAC improves the throughput of best-effort applications by 3.87× while ensuring the required QoS of latency-critical applications.

The technologies of heterogeneous multi-core architectures, co-location, and virtualization can be used to reduce server power consumption and improve system utilization, which are three important technologies for data centers. This article explores the scheduling strategy of Emulator threads within virtual machine processes in a scenario of co-location of multiple virtual machines on heterogeneous multi-core architectures. In this co-location scenario, the scheduling strategy for Emulator threads significantly affects the performance of virtual machines. This article focuses on this thread for the first time in the relevant field. This article found that the scheduling latency metric can well indicate the running status of the vCPU threads and Emulator threads in the virtualization environment, and applied this metric to the design of the scheduling strategy. This article designed an Emulator thread scheduler based on heuristic rules, which, in coordination with the host operating system's scheduler, dynamically adjusts the scheduling scope of Emulator threads to improve the overall performance of virtual machines. The article found that in real application scenarios, the scheduler effectively improved the performance of applications within virtual machines, with a maximum performance improvement of 40.7%.

In-memory key-value stores (IMKVSes) serve many online applications. They generally adopt the fork-based snapshot mechanism to support data backup. However, this method can result in query latency spikes because the engine is out-of-service for queries during the snapshot. In contrast to existing research optimizing snapshot algorithms, we address the problem from the operating system (OS) level, while keeping the data persistent mechanism in IMKVSes unchanged. Specifically, we first study the impact of the fork operation on query latency. Based on findings in the study, we propose Asyncfork, which performs the fork operation asynchronously to reduce the out-of-service time of the engine. Async-fork is implemented in the Linux kernel and deployed into the online Redis database in public clouds. Our experiment results show that Async-fork can significantly reduce the tail latency of queries during the snapshot.

Multi-threading is a common way for programs to benefit from the multi/many-core design. However, the performance of some parallel programs does not increase/even decrease as the number of cores/threads increases. Our study shows that the performance of a parallel program is impacted by the number of cores/threads , the thread placement , the inputs of the program . It is nontrivial to identify the optimal number of cores and the corresponding thread placement to maximize the performance, when the input of a program is determined online and the workload of different iterations may not be identical. To resolve the above problem, we propose Otter , a thread auto-tuning system at runtime for iterative parallel programs. Otter collects the runtime information in the first few iterations and makes decisions on the number of threads and thread placement policy to achieve the goal of improving performance or saving resources. It considers the characteristics of dynamic workload in the iteration process and reduces the time overhead through a migration method. Experiments on a 96-core machine show that Otter improves the performance of the benchmarks by 20.7% and reduces core hours by 51.3% on average compared to the case of running them with all the CPU cores.

I/O-intensive applications are important workloads of public clouds. Multiple cloud applications co-run on the same physical machine in different virtual machines (VMs), and the shared resources (e.g., disk bandwidth) are often isolated for fairness. Our investigation shows that the performance of an I/O-intensive application is impacted by both disk bandwidth allocation and the page cache settings in the guest operating system. However, none of prior work considers adjusting the page cache settings for better performance, when the disk bandwidth allocation is adjusted. We therefore propose CSC, a system that collaboratively identifies the appropriate disk bandwidth allocation and page cache settings in the guest operating system of each VM. CSC aims to improve the system-wide I/O throughput of the physical machine, while also improve the I/O throughput of each individual I/O-intensive application in VMs. CSC comprises an online disk bandwidth allocator and an adaptive dirty page setting optimizer. The bandwidth allocator monitors the disk bandwidth utilization and re-allocates some bandwidth from free VMs to busy VMs periodically. After the re-allocation, the optimizer identifies the appropriate dirty page settings in the guest operating system of the VMs using Bayesian Optimization. The experimental results show that CSC improves the performance of I/O-intensive applications by 9.5% on average (up to 17.29%) when 5 VMs are co-located while fairness is guaranteed.

Large-scale datacenters often host latency-sensitive services that have stringent Quality-of-Service requirement and experience diurnal load pattern. Co-locating best-effort applications that have no QoS requirement with the latency-sensitive services has been widely used to improve the resource utilization of datacenters with careful shared resource management. However, existing co-location techniques tend to result in the power overload problem on power constrained servers due to the ignorance of the power consumption. To this end, we propose Sturgeon, a runtime system proactively manages resources between co-located applications in a power constrained environment, to ensure the QoS of latency-sensitive services while maximizing the throughput of best-effort applications. Our investigation shows that, at a given load, there are multiple feasible resource configurations to meet both QoS requirement and power budget, while one of them yields the maximum throughput of best-effort applications. To find such a configuration, we establish models to accurately predict the performance and power consumption of the co-located applications. Sturgeon monitors the QoS of the services periodically, in order to eliminate the potential QoS violation caused by the unpredictable interference. Besides, when the datacenter hosts different types of applications to perform co-location, Sturgeon places applications with their preferable candidates to improve the overall throughput. The experimental results show that at server level Sturgeon improves the throughput of the best-effort application by 25.43 percent compared to the state-of-the-art technique, while guaranteeing the 95%-ile latency within the QoS target; at cluster level, Sturgeon improves the overall throughput of best-effort applications by 13.74 percent compared to the baseline.

Large-scale datacenters often host latency-sensitive services that have stringent Quality-of-Service requirement and experience diurnal load pattern. Co-locating best-effort applications that have no QoS requirement with latency-sensitive services has been widely used to improve the resource utilization with careful shared resource management. However, existing co-location techniques tend to result in the power overload problem on power constrained computers due to the ignorance of the power consumption. To this end, we propose Sturgeon, a runtime system proactively manages resources between colocated applications in a power constrained environment, to ensure the QoS of latency-sensitive services while maximizing the resource utilization. Our investigation shows that, at a given load, there are multiple feasible resource configurations to meet both QoS requirement and power budget, while one of them yields the maximum throughput of best-effort applications. To find such a configuration, we establish models to accurately predict the performance and power consumption of the colocated applications. Sturgeon monitors the QoS periodically in order to eliminate the potential QoS violation caused by the unpredictable interference. The experimental results show that Sturgeon improves the throughput of best-effort applications by 24.96% compared to the state-of-the-art technique, while guaranteeing the 95%-ile latency within the QoS target.

Monolithic three-dimensional integration (M3I) directly fabricates tiers of integrated circuits upon each other and provides millions of vertical interconnections with inter-layer vias (ILVs). It thus brings higher integration density and communication capability compared with three-dimensional stacked integration (3D-SI). However, the Known-Good-Die problem haunting 3D-SI-a faulty tier causes the failure of the entire stack-also occurs in M3I. Lack of efficient test methodologies such as the pre-bond testing in 3D-SI, M3I may have a more significant yield drop and thus its cost may be unacceptable for main-stream adoption. This paper introduces a novel In-growth test method for M3I SRAM. We propose a novel Design-for-Test (DfT) methodology to enable the proposed In-growth test on cell-level partitioned incomplete SRAM cells. We also build a statistical model of cost and discover a prospective judgement to determine whether or not to stop the fabrication, in order to prevent from raising the cost of fabricating more tiers upon the irreparable tiers. We find that a “sweet point” exists in the judgement, which can minimize the overall cost. Experimental results show the effectiveness of our proposed test methodology.

We proposed a novel pre-bond functional test architecture for core -level partitioned 3D SoCs. In this test architecture, a new test access mechanism is built by probing functional Through Silicon Vias (TSVs) and by linking functional TSVs to all pins of IP cores under test. In order to reduce the Design-for-Test (DfT) cost, we propose a novel scheme to share the functional TSVs among pins of IP cores and reuse wires which connect pins to the same functional TSVs. Experimental results based on the MCNC benchmarks circuits demonstrate the feasibility and effectiveness of the DfT optimization methods.

In three-dimensional (3D) integrated circuits (IC-s), many clock-TSVs are deployed to deliver clock signals to different tiers with minimum skews. However, these clock-TSVs are prone to aging effects, such as thermal-mechanical stress and electromigration, rendering hard-to-predict clock skews at runtime. These skews have a wide range of influence on the flip-flops, and may violate the safety margins of critical paths in the circuit. Besides the circuit aging effect, the clock-TSV induced skews pose another threat to the circuit lifetime reliability. To tackle this problem, we propose to put tunable buffer for each clock-TSV in the clock network, and introduce an efficient algorithm to place aging sensors in the circuit at design stage. Then, at runtime, we conduct online diagnosis and apply effective clock tuning algorithms based on the triggered alarms in the aging sensors. Experimental results on a post-layout 3D circuit show that the proposed solution is able to significantly improve the lifetime reliability of 3D ICs.