The relentless growth of sizes and traffic volumes in data center networks (DCN) is posing a significant challenge on switching: A single giant switch in a modern flattened-topology DCN needs to direct terabits of traffic per second to hundreds of top-of-rack switches in a low latency of just a few milliseconds. The vast majority of existing switching schedulers have yet to address this challenge: Traffic-oblivious RODCN (reconfigurable optical DCN) schedulers do not have high throughput utilization, whereas on-demand hybrid (circuit and packet) switching schedulers are too computationally expensive. To this end, we propose an efficient and performant hybrid switching scheduler, named QPS-Fit, which computes high-quality on-demand schedules in a fully parallelizable way. Our algorithm simultaneously fits many transmissions into the schedule, all by parallel "dirt-cheap" request-grant message exchanges between input and output ports. According to our simulation, QPS-Fit achieves similar throughput utilization as BFF, the state-of-the-art on-demand hybrid switching scheduler, while being nearly 27 x faster when running on 96 threads.
The rapid explosion of Artificial Intelligence (AI) workloads utilizing a growing number of accelerators has placed unprecedented demand on the network. These workloads typically leverage Remote Direct Memory Access (RDMA) and require a high-performance network fabric. While many purposebuilt cloud networking solutions can provide high performance, efficiently utilizing these costly infrastructures require a fabric that is multi-tenant for ease of consumption. Furthermore, the fabric must be resilient to faults and for operational manageability. Resilient cloud networks typically employ mature Ethernet segmentation techniques over Clos topologies with Equal Cost Multi Path (ECMP) routing. ECMP hashes flows to paths, which in case of collisions can significantly degrade performance for large RDMA over Converged Ethernet (RoCE) flows. To mitigate ECMP penalties, we evaluate routing strategies with varying levels of operational complexity. We explore load balancing and path pinning solutions that leverage non-proprietary, mature technologies over commodity Ethernet. Our evaluation follows a three-fold strategy, focusing on the key dimensions of performance, resiliency, and operational complexity. By applying this methodology to representative implementations, we highlight the trade-offs. While all techniques are resilient, path pinning-based solutions excel at performance but introduce greater complexity. Specifically, path pinning achieves up to 1.6x improvement over ECMP for RoCE test traffic and up to 2.5x for NCCL AllReduce. These results validate the promising performance benefits of path pinning and highlight the need to explore less complex implementations for broader adoption. Our methodology can be used to rigorously evaluate future implementations in support of AI network design.
Managing diverse data formats and improving query processing efficiency at cloud service centers is crucial in the era of interconnected heterogeneous devices. Lakehouse effectively manages heterogeneous data but faces challenges in maintaining SQL query performance and data independence in large-scale analyses. Precomputation, which stores intermediate results in advance, reduces query latency by a space-for-time trade-off. This paper addresses the challenges of matching and rewriting results in lakehouse environments. By combining pre-computation with the low-cost and dynamically scalable storage characteristics of cloud storage, we introduce an optimized lakehouse architecture with an in-memory index and dynamic task scheduling to enhance OLAP performance while balancing data center resources. Experiments demonstrate improvements in query performance ranging from 6.1 % to 64.9% over native lakehouse architectures and reductions in resource occupation from the original 75.6% to 17.3% with preloading and 25.0% with dynamic scheduling.
Change point detection (CPD) is crucial for identifying abrupt shifts in time series data across various domains. In microservice architectures, where components operate at high velocity and scale, monitoring efficiency becomes particularly critical for maintaining system reliability while minimizing overhead. Efficient change point detection allows for timely root cause analysis, dynamic resource allocation, and targeted instance restarts, interventions that can prevent cascading failures across interconnected services. This paper presents a novel approach called MMR-FFT CPD that transforms high-dimensional microservice telemetry data into a one-dimensional kernel-based representation through Maximum Margin Regression (MMR). The resulting 1D time series is then processed using Fast Fourier Transform (FFT) for efficient temporal pattern forecasting, which enables prediction of expected future values. By computing the correlation between these FFT-based predictions and the actual observed values, our method can identify deviations that indicate potential change points in microservice behavior. By tracking only these compact representations rather than raw metrics, our method dramatically reduces computational overhead and network traffic in distributed monitoring scenarios. We evaluated MMR-FFT CPD against multiple state-of-the-art algorithms. Our experiments demonstrate that with a 1D data transfer to the monitoring entity, we can maintain competitive detection accuracy, which is advantageous, particularly for highdimensional data streams typical in modern distributed systems. The results provide valuable insights for implementing efficient, low-overhead monitoring in microservice environments with varying computational and network constraints, enabling more responsive and resilient microservice architectures.
Recent advances in multi-core chip technology have enabled the dynamic tuning of shared memory resources, such as last-level cache and memory bus bandwidth. However, despite proven performance benefits, the complexity of effectively utilizing these hardware-level QoS enforcement features has limited their adoption in real-world cloud computing environments. In this paper, we introduce ESTHER, a novel approach to autonomously fine-tune QoS enforcement features in cloud environments using extremum seeking control, focusing on applications needs and operator ease-of-use. We demonstrate that ESTHER effectively maintains latency-critical workload SLOs and rapidly resolves any infringements by prioritizing shared memory resources. Such fast node-level resolution of SLO violations ensures that costly cluster-level scaling events may be avoided. Furthermore, ESTHER improves best-effort job throughput without impacting latency-critical workloads, achieving performance gains without utilizing workload profiling or prior knowledge of system dynamics.
High-Performance Computing systems frequently undergo architectural changes, such as hardware upgrades or the deployment of new clusters, to meet evolving computational demands. Traditional static schedulers and machine-learning-based approaches struggle to adapt efficiently to these changes, often requiring manual adjustments or extensive retraining. In this paper, we propose a novel approach combining Separate Feature Extraction and Selective Transfer Learning to enable rapid adaptation of Reinforcement Learning-based HPC schedulers to new or modified cluster architectures. We evaluate our approach using three real-world HPC clusters, including both CPU and GPU architectures. Our experiments simulate scheduler transitions between these clusters, capturing a wide range of architectural changes and workload variations found in practice. Applied to a state-of-the-art hierarchical RL-based scheduler, our method demonstrates rapid adaptation across diverse system configurations and workloads. In the six transition experiments, the schedulers enhanced with our approach not only quickly outperformed all 21 baseline heuristic methods in terms of jobs' average and maximum waiting time but also achieved performance comparable to schedulers trained from scratch. Notably, even in the most time-consuming transition case, integrating our approach reduced the required training timesteps to just 1.76% of the total timesteps needed for training a scheduler from scratch - representing a 56.8x reduction in training effort. This demonstrates its ability to efficiently adapt existing schedulers to evolving HPC architectures with minimal cost, providing a practical solution for real-world HPC operations.
Modern software development abstracts applications from the underlying infrastructure, enabling global-scale deployment with minimal concern about low-level networking details. However, when these infrastructure-agnostic software components need to communicate, they encounter significant networking limitations. This forces developers to either navigate complex, low-level networking constructs to achieve the desired connectivity or give up on truly flexible connectivity and limit their software to static connectivity patterns. In this paper, we focus on the evolving challenges of application connectivity in today's hyper-distributed reality. We propose to model connectivity around the notion of application services and have realized this proposal as ClusterLink, which exposes the app-level APIs for specifying communication policies at a very granular level and implements them efficiently. This paper shares details on ClusterLink design principles, APIs, architecture, and implementation, and shows that ClusterLink outperforms its closest competitor by 2.5x in throughput in a cloud-based experimental setting.
This paper presents SSL-S3, a novel approach that revisits SQL Statement Logging to fully exploit the newly introduced S3 append feature. When a transaction commits, SSL-S3 captures update SQL statements and appends them to S3. Unlike conventional physical logging such as Snapshot and Incremental logging, which incur redundant writes by storing physical data changes, SSL-S3 appends only update statements to S3. Our experimental results demonstrate that SSL-S3 outperforms Snapshot logging by up to 3.5x, and Incremental logging by 1.4x. Moreover, SSL-S3 reduces the recovery time by a factor of 5 compared to Incremental logging and by one-sixth compared to Snapshot logging.
Traffic scheduling for microservices in emerging edge-cloud environments is challenged by heterogeneous node performance and varying inter-node latencies. Inefficient scheduling among replicas may lead to replica overload and excessive communication overhead, ultimately degrading Quality of Service (QoS) metrics-specifically, the 99th percentile tail latency(P99 latency). To address these challenges, we introduce HEART, a novel two-stage traffic scheduler that jointly accounts for node heterogeneity and network latency. In Stage 1, HEART computes per-replica traffic proportions using a sliding window and an exponentially weighted moving average of CPU usage and request rates, thereby capturing recent load trends. In Stage 2, HEART applies k-means clustering to inter-node latency measurements to identify and prune high-latency links, which in turn reduces communication delays. A Maximum Flow algorithm is subsequently employed to verify whether the pruned network supports the required traffic flow; if the reduced network proves insufficient, additional links are incrementally reinstated until a feasible configuration is achieved. Finally, a Minimum-Cost Flow algorithm-selected for its proven optimality in network flow allocation-is applied to distribute traffic cost-effectively across the network. Experimental results demonstrate that HEART significantly reduces the P99 latency compared to existing approaches such as the default Kubernetes scheduler, the Least- Request algorithm of Istio, OptTraffic, and LATA, thereby enhancing overall QoS.
Cloud forensics is a crucial yet challenging field, as traditional forensic techniques struggle to handle the large-scale, dynamic nature of cloud environments. Manual forensic analysis is time-consuming, error-prone, and often fails to detect evolving cyber threats. This paper presents a novel tool leveraging Large Language Models (LLMs) to fully automate cloud forensic investigations. Our approach utilizes few-shot learning to classify log data, extract forensic intelligence, and reconstruct attack timelines. We evaluate LLM-based automation against traditional machine learning models, including Random Forest, XGBoost, and Gradient Boosting, using cloud forensic log datasets. Experimental results demonstrate that LLMs improve forensic accuracy, precision, and recall while reducing the need for extensive feature engineering. However, challenges such as hallucination risks, adversarial manipulation, and forensic explainability must be addressed to ensure the reliability of AI-driven investigations. To mitigate these risks, we explore Retrieval-Augmented Generation (RAG) for context-aware forensic intelligence and propose hybrid AI models integrating rule-based forensic validation. Our findings highlight the potential of LLM-driven forensic automation to enhance cloud security operations while outlining key areas for future research, including adversarial robustness, forensic transparency, and multi-cloud scalability.
The rise of microservice architecture in cloud computing has introduced additional complexities in diagnosing faults, as traditional end-to-end tracing systems often fail to address issues beyond the application layer, such as network devices. To overcome this limitation, we introduce TraceWizard, an enhanced end-to-end tracing system that integrates eBPF and SDN (Software Defined Network) technologies to track requests across host and network devices. By enabling full life-cycle tracing and maintaining consistent trace contexts, TraceWizard provides fine-grained insights into faults across applications, the OS kernel, and network devices. Our evaluations show that its data enables more effective fault detection, achieving an average accuracy of 91.6 % across different algorithms—significantly outperforming application-layer monitoring tools. Additionally, it helps operators identify root causes with minimal overhead, reducing QPS by 2.2%, increasing QCT by 2.2%, and adding 3.41 % CPU and 2.43% memory usage.
Serverless cloud computing is increasingly adopted for workflow management, optimizing resource utilization for providers while lowering costs for customers. Integrating edge computing into this paradigm enhances scalability and efficiency, enabling seamless workflow distribution across geographically dispersed resources on the computing continuum. However, existing serverless workflow orchestration methods on the computing continuum often prioritize time and/or cost objectives, neglecting energy consumption and carbon footprint. This paper introduces EnergyLess, a multi-objective concurrent serverless workflow batch orchestration service for the computing continuum. EnergyLess decomposes workflow functions within a batch into finer-grained sub-functions and schedules either the original or sub-function versions to suitable regions and instances on the continuum, improving energy consumption, carbon footprint, economic cost, and completion time while considering individual workflow requirements and resource constraints. We formulate the problem as a mixed-integer nonlinear programming (MINLP) model and propose three lightweight heuristic algorithms to enable scalable function scheduling and execution. Evaluations on a large-scale computing continuum testbed, spanning AWS Lambda, Google Cloud Functions (GCF), and 325 fog and edge instances across six regions, demonstrate that EnergyLess improves cost efficiency by 75%, completion time by 6%, energy consumption by 15%, and CO2 emissions by 20% for a batch size of 300, compared to three baseline methods.
Deep Neural Network (DNN) inference serving presents significant challenges due to variable workloads, heterogeneous models, and strict Service Level Objectives (SLOs). While GPU acceleration enables high-throughput inference, the cost-effectiveness of these specialized resources depends on maintaining high utilization. Current DNN serving systems employ static or heuristic-based batching strategies that cannot adapt effectively to dynamic cloud workloads. We present DNN-Adapt, a novel system that employs reinforcement learning (RL) to dynamically optimize batching decisions in DNN serving environments. DNN-Adapt introduces a hybrid batching framework that combines traditional batching techniques with RL-based decision making, a multi-timescale architecture that operates at different levels of granularity, and a hybrid decision system that integrates rule-based safety constraints with learned policies.
Serverless edge computing improves performance by processing data closer to its source, reducing operational costs, and increasing server utilization. Despite these benefits, edge servers face scalability challenges and queuing delays due to limited resources. Horizontal offloading can alleviate excessive workloads by efficiently distributing tasks across edge servers. However, it introduces higher waiting time, cold starts latency, and missed task deadlines at the receiving edge servers. To address these challenges, we introduce PROBA, which utilizes a Double Dueling Deep Q-learning algorithm and In-node scheduling that optimize the task offloading between the edge servers and improve task scheduling within the edge server. The approach uses probabilistic resource sharing, where edge servers share their real-time availability to a central cloud system. The cloud analyzes these performance metrics based on user-specified rewards to determine optimal scheduling decisions, which the edge servers execute to maintain balanced and responsive work-load distribution. We evaluated PROBA in a serverless edge computing simulator that focuses on horizontal offloading and in-node scheduling. In the evaluation using real-world trace data from Alibaba, our PROBA technique decreased the average wait time from 3.1 s to below 0.69 $s$. PROBA also gave 1.37 % better task completion time than competitors.
The use of microservices-based architectures is becoming more prominent due to their advantageous characteristics, such as manageability, scalability, and flexibility. However, their management can be complex, and their performance can be affected by high latencies, which can alter the Service Level Objective (SLO). In order to identify the causes of high latencies, we present a causal modelling framework which is capable of analysing and reconstructing latencies within microservice-based architectures. To this end, we employ causal discovery to identify the causes of latencies. Our model integrates domain knowledge to impose constraints on the causal graph, ensuring the accuracy of the discovered relationships as well as accelerating the causal discovery. To validate our approach, we reconstruct latency metrics using machine learning techniques and demonstrate the effectiveness of our approach by accurately capturing the interrelationships between microservice resources. Our framework provides a better understanding of the causes of latencies that lead to SLO violations, and paves the way for sophisticated mechanisms that enable proactive management of cloud resources.
The rapid advancement of LLMs has led to widespread adoption across various domains, but it has also raised concerns about data security and privacy, particularly with publicly available and commercially operated platforms. Given their high computational demands, cloud environments are the obvious choice for deployment. As a result, organizations are increasingly deploying LLMs in confined cloud environments to protect sensitive data while leveraging scalable cloud resources. However, deploying LLMs in cloud environments remains a complex and time-consuming process that requires specialized skills and expertise in various areas, such as infrastructure management, resource allocation, and model setup. Testing and comparing LLMs to select the appropriate one is particularly challenging as different models are trained for different purposes, making the direct comparison nontrivial. Furthermore, differences in model architectures, training data, and finetuning strategies make objective evaluation difficult, limiting the effectiveness of traditional benchmarking approaches. To address these challenges, we present a cloud-native system that automates both the deployment and evaluation of LLMs. Our contributions are twofold: (i) we automate the provisioning and deployment of LLMs on various cloud platforms to streamline infrastructure setup, and (ii) we develop a lightweight evaluation framework that leverages the LLM-as-a-Judge approach, where an independent LLM systematically assesses and compares different models based on predefined evaluation criteria. Our ongoing work aims to optimize LLM deployment by selecting cost-efficient cloud resources. We are also enhancing the evaluation framework with diverse prompts, broader metrics, and cross-model validation for fair, reproducible benchmarking.
Serverless technologies have simplified distributed computing by streamlining resource management and offering out-of-the-box usability of cloud resources. However, serverless computing has yet to fully permeate the broader data analytics community. One of the major reasons causing this slow adoption is the lack of a handy serverless interface for seamlessly running recurrent workloads in the cloud. To fill this gap, we introduce in this work the missing piece in serverless analytics: the Ortzi Dataframe, a practical and intuitive programming abstraction that mirrors pandas DataFrames, so that users can effortlessly run their local, single-threaded Python code at scale in the cloud. Needless to say, such a powerful abstraction is certainly useless if not backed by a serverless analytics system that can operate over it in parallel. For this reason, another major contribution of this paper is a fully-fledged system that can run jobs in parallel across the cloud continuum using the novel Ortzi Dataframes. The new system leverages the specific capabilities of each serverless backend without user intervention. Our evaluation demonstrates that Ortzi enables exploration of the nuanced trade-offs of heterogeneous backends with min-imal programming changes and overhead. By harnessing the seamless nature of Ortzi, we optimize jobs through strategic backend selection, still delivering a user-friendly open source framework for programmers without cloud expertise.
LLM inference is essential for applications like text summarization, translation, and data analysis, but the high cost of GPU instances from Cloud Service Providers (CSPs) like AWS is a major burden. This paper proposes InferSave, a cost-efficient VM selection framework for cloud-based LLM inference. InferSave optimizes KV cache offloading based on Service Level Objectives (SLOs) and workload characteristics, estimating GPU memory needs, and recommending cost-effective VM instances. Additionally, the Compute Time Calibration Function (CTCF) improves instance selection accuracy by adjusting for discrepancies between theoretical and actual GPU performance. Experiments on AWS GPU instances show that selecting lower-cost instances without KV cache offloading improves cost efficiency by up to 73.7% for online workloads, while KV cache offloading saves up to 20.19% for offline workloads.
In the rapidly evolving landscape of cloud-native computing, Organizations are increasingly adopting infrastructure models that emphasize scalability, flexibility, and efficiency. Kubernetes has become the de facto standard for orchestrating containerized applications in these environments. However, the inherent complexity of cloud-native ecosystems introduces significant challenges, particularly in the form of misconfigurations that can compromise both security and performance. This study explores the potential of Large Language Models (LLMs) in identifying Kubernetes misconfigurations. We introduce a comprehensive taxonomy of common misconfiguration types, offering a structured framework to better understand and categorize these issues. Additionally, we conduct an empirical evaluation of state-of-the-art detection tools to benchmark their effectiveness. Furthermore, we analyze the Kubernetes objects most prone to misconfiguration and evaluate the severity of the identified issues. By leveraging advanced machine learning techniques, including LLMs, we provide novel insights into enhancing misconfiguration detection methodologies.
Modern cloud applications consist of containerized microservices deployed to a virtual computing environment, such as a Kubernetes cluster. Policies are needed to block unintended and potentially harmful interactions between the microservices, which attackers could exploit for discovery and lateral movement. However, it is challenging for cluster administrators to define strict network policies because the interactions between the components are not clearly defined. Enabling them often requires manual inspection of the declarative configuration and the source code of the applications. This paper proposes a novel approach to creating Kubernetes network policies that restricts access between microservices within a cluster. Based on the principles of modularity and information hiding, we identify service composition patterns in cloud applications and use them to create network policies. The policy generation is implemented as an open-source tool, Helm-ET, which we evaluate on 451 Helm charts across three datasets. The results show that the proposed approach can significantly reduce the internal attack surface in the Kubernetes cluster (92.71% less allowed connections), achieving comparable results to state-of-the-art tools. However, compared to other solutions, Helm-ET is faster (< 100ms vs 79 seconds) and the policies before the application deployment.
