Current quantum processors are too limited to solve large combinatorial optimization problems directly, motivating hybrid executions that combine quantum processors with classical HPC resources. This paper presents QSplit, a workflow-oriented hybrid quantum-classical framework for distributed optimization. QSplit targets problems expressed in the Quadratic Unconstrained Binary Optimization form, decomposes large instances into smaller sub-instances, dispatches them to heterogeneous classical or quantum backends, and reconstructs a global solution within a portable and reproducible workflow. The orchestration layer of the QSplit workflow is managed by StreamFlow WMS, enabling configurable execution across cloud, HPC, and remote quantum infrastructures. We evaluate QSplit on Knapsack and Max-Cut instances to study how problem structure affects decomposition quality. Results show that QSplit achieves competitive behavior on sparse instances, where decomposition preserves enough structure for effective aggregation, while dense instances expose the limits of purely structural splitting due to the loss of global correlations. These findings suggest that QSplit provides a framework for studying hybrid HPC-quantum optimization pipelines and their current scalability boundaries.
Achieving peak GPU performance remains a significant challenge as the system throughput is constrained by host-device synchronization delays and kernel scheduling overheads, even with aggressive kernel optimizations and batch processing. Furthermore, existing approaches often underutilize hardware resources such as compute cores and copy engines due to scheduling overheads. To address these problems, we propose a CUDA runtime framework for task-parallel pipelines to minimize the synchronization overheads and the gap between kernel executions. The proposed solution combines two innovations: (1) a multi-stream task-parallel pipeline programming model that leverages event-chaining and work-stealing mechanisms to fully utilize available hardware resources; (2) a graph-based execution flow with per-stream buffers to ensure memory safety for multiple in-flight jobs running concurrently. Extensive evaluations on representative real-world workloads show 1.15–1.44 × speedup and reduce scheduling overheads by 18–54
The currently dominant AI/ML workloads, such as Large Language Models (LLMs), rely on the efficient execution of General Matrix-Matrix Multiplication (GEMM) operations. Thus, most systems are equipped with dedicated matrix hardware accelerators built on square Systolic Arrays (SAs) of Processing Elements (PEs). While this organization was effective for traditional Deep Neural Networks (DNNs), LLMs introduce input-dependent and highly skewed matrices, leading to underutilized SA resources. To address this challenge, we propose SISA (Scale-In Systolic Array), a novel SA architecture that partitions the traditional square array into horizontal rectangular slabs. With minimal overhead, SISA exposes parallelism through independently scheduled slabs for efficient execution of small or skewed matrix shapes, while retaining full-array operation for large GEMMs. SISA achieves up to 8.52 × speedup and 93
Training large language models (LLMs) on heterogeneous GPU clusters is increasingly common, but resource provisioning remains a core bottleneck. Existing systems either require users to manually choose GPU types and counts or rely on static memory formulas that miss runtime factors (e.g., allocator behavior and communication buffers), leading to OOM failures or conservative over-provisioning. At the same time, heterogeneous scheduling often depends on expensive optimization procedures that introduce high scheduling latency. To solve this problem, we propose Frenzy, a memory-aware serverless LLM training system for heterogeneous GPU clusters. Frenzy first estimates peak training memory using analytical modeling calibrated by runtime feedback to infer robust GPU type-and-quantity plans. It then applies a pricing-driven greedy scheduler with lightweight local swaps to achieve efficient near-optimal placement with low overhead. We evaluate Frenzy using multi-task LLM training on a physical heterogeneous cluster with 8 nodes and 4 GPU types. Compared with state-of-the-art baselines, Frenzy reduces average job completion time by 12 × , and achieves over 97
Alpine livestock monitoring is critical for ecological preservation and agricultural efficiency. However, existing solutions struggle with energy constraints, limited network availability, and intermittent connectivity in remote environments. To address this, we propose an on-device federated learning framework tailored for PV-powered IoT sensors to optimize energy-communication tradeoffs. Our approach introduces staleness-aware aggregation and solar-aware training scheduling to address intermittent connectivity and PV variability in remote alpine environments. Deployed on a real-world testbed with collar sensors, the framework achieves 92
