Large Language Models (LLMs) are increasingly deployed in multi-agent systems, where effective inter-model communication is crucial. Existing communication protocols either rely on natural language, incurring high inference costs and information loss, or on hidden states, which suffer from information concentration bias and inefficiency. To address these limitations, we propose KVComm, a novel communication framework that enables efficient communication between LLMs through selective sharing of KV pairs. KVComm leverages the rich information encoded in the KV pairs while avoiding the pitfalls of hidden states. We introduce a KV layer-wise selection strategy based on attention importance scores with a Gaussian prior to identify the most informative KV pairs for communication. Extensive experiments across diverse tasks and model pairs demonstrate that KVComm achieves comparable performance to the upper-bound method, which directly merges inputs to one model without any communication, while transmitting as few as 30\% of layers' KV pairs. Our study highlights the potential of KV pairs as an effective medium for inter-LLM communication, paving the way for scalable and efficient multi-agent systems.

Large Language Model (LLM) inference is rapidly becoming a core datacenter service, yet current serving stacks keep the host CPU on the critical path for orchestration and token-level control. This makes LLM performance sensitive to CPU interference, undermining application colocation and forcing operators to reserve CPU headroom, leaving substantial capacity unutilized. We introduce Blink, an end-to-end serving architecture that removes the host CPU from the steady-state inference path by redistributing responsibilities across a SmartNIC and a GPU. Blink offloads request handling to the SmartNIC, which delivers inputs directly into GPU memory via RDMA, and replaces host-driven scheduling with a persistent GPU kernel that performs batching, scheduling, and KV-cache management without CPU involvement. Evaluated against TensorRT-LLM, vLLM, and SGLang, Blink outperforms all baselines even in isolation, reducing pre-saturation P99 TTFT by up to 8.47× and P99 TPOT by up to 3.40×, improving decode throughput by up to 2.1×, and reducing energy per token by up to 48.6%. Under CPU interference, Blink maintains stable performance, while existing systems degrade by up to two orders of magnitude.

LLM training at the scale of tens of thousands of GPUs now spans multiple datacenters (DC), making cross-DC collectives over long-haul links unavoidable. A critical and overlooked bottleneck arises when these collectives collide with intra-DC traffic at the destination - a common pattern in real workloads. The multi-millisecond congestion control loop is too slow to react, triggering severe packet loss and congestion collapse. We present Spillway, a transparent in-network mechanism that buffers dropped packets in switch-disaggregated buffers in a destination data center and drains them once congestion subsides. Through large-scale end-to-end simulations and a hardware prototype, we show that Spillway eliminates performance degradation from collective collisions, reducing iteration time by up to 14

Novel backscatter communication techniques allow battery-free sensor tags to operate with standard IoT devices, thereby augmenting a network's sensing capabilities. For communicating, sensor tags rely on an unmodulated carrier provided by neighboring IoT devices, with a schedule coordinating this provisioning across the network. Computing schedules to interrogate all sensor tags while minimizing energy, spectrum utilization, and latency-i.e., carrier scheduling-is an NP-hard problem. While recent work introduces learning-based systems for carrier scheduling, we find that their advantage over traditional heuristics progressively decreases for networks with hundreds of IoT nodes. Moreover, we find that their generalization is not consistent: it greatly varies across identically trained models while fixing the dataset, hyperparameters and random seeds used. We present RobustGANTT, a Graph Neural Network scheduler for backscatter networks that learns from optimal schedules of small networks (up to 10 nodes). Our scheduler generalizes, without the need for retraining, to networks of up to hundreds of nodes ( 100x training topology sizes), and exhibits consistent generalization across independent training rounds. We evaluate our system on both simulated topologies of up to 1000 nodes and real-life IoT network topologies of up to 300 IoT devices. RobustGANTT not only exhibits better generalization than existing systems, it also computes schedules achieving up to 2x less energy and spectrum utilization. Additionally, its polynomial runtime complexity allows it to react fast to changing network conditions. Our work facilitates the operation of large-scale IoT networks, and our machine learning findings further advance the capabilities of learning-based network scheduling. We release our code, datasets and pre-trained models.

Efficiently managing compute resources for Large Language Model (LLM) inference remains challenging due to the inherently stochastic and variable lengths of autoregressive text generation. Accurately estimating response lengths in advance enables proactive resource allocation, yet existing approaches either bias text generation towards certain lengths or rely on assumptions that ignore model- and prompt-specific variability. We introduce CASTILLO, a dataset characterizing response length distributions across 13 widely-used open-source LLMs evaluated on seven distinct instruction-following corpora. For each ⟨prompt, model⟩ sample pair, we generate 10 independent completions using fixed decoding hyper-parameters, record the token length of each response, and publish summary statistics (mean, std-dev, percentiles), along with the shortest and longest completions, and the exact generation settings. Our analysis reveals significant inter- and intra-model variability in response lengths (even under identical generation settings), as well as model-specific behaviors and occurrences of partial text degeneration in only subsets of responses. CASTILLO enables the development of predictive models for proactive scheduling and provides a systematic framework for analyzing model-specific generation behaviors. We publicly release the dataset and code to foster research at the intersection of generative language modeling and systems.

AI workloads, often hosted in multi-tenant cloud environments, require vast computational resources but suffer inefficiencies due to limited tenant-provider coordination. Tenants lack infrastructure insights, while providers lack workload details to optimize tasks like partitioning, scheduling, and fault tolerance. We propose HarmonAIze to redefine cloud abstractions, enabling cooperative optimization for improved performance, efficiency, resiliency, and sustainability. We outline key opportunities and challenges this vision faces.

Formal verification is increasingly recognized as a critical foundation for building reliable software systems. However, the need for specialized expertise to write precise specifications, navigate complex proof obligations, and learn annotations often makes verification an order of magnitude more expensive than implementation. While modern AI systems can recognize patterns in mathematical proofs and interpret natural language, effectively integrating them into the formal verification process remains an open challenge. We present Prometheus, a novel AI-assisted system that facilitates automated code verification with current AI capabilities in conjunction with modular software engineering principles (e.g., modular refactoring). Our approach begins by decomposing complex program logic, such as nested loops, into smaller, verifiable components. Once verified, these components are recomposed to construct a proof of the original program. This decomposition-recomposition workflow is non-trivial. Prometheus addresses this by guiding the proof search through structured decomposition of complex lemmas into smaller, verifiable sub-lemmas. When automated tools are insufficient, users can provide lightweight natural language guidance to steer the proof process effectively. Our evaluation demonstrates that transiently applying modular restructuring to the code substantially improves the AI's effectiveness in verifying individual components. This approach successfully verifies 86

Large Language Models (LLMs) have demonstrated their exceptional performance in various complex code generation tasks. However, their broader adoption is limited by significant computational demands and high resource requirements, particularly memory and processing power. To mitigate such requirements, model pruning techniques are used to create more compact models with significantly fewer parameters. However, current approaches do not focus on the efficient extraction of programming-language-specific sub-models. In this work, we explore the idea of efficiently deriving coding-specific sub-models through unstructured pruning (i.e., Wanda). We investigate the impact of different domain-specific calibration datasets on pruning outcomes across three distinct domains and extend our analysis to extracting four language-specific sub-models: Python, Java, C++, and JavaScript. We demonstrate that it is possible to efficiently extract programming-language-specific sub-models using appropriate calibration datasets while maintaining acceptable accuracy w.r.t. full models. We are also the first to provide analytical evidence that domain-specific tasks activate distinct regions within LLMs, supporting the creation of specialized submodels through unstructured pruning. We believe that this work has significant potential to enhance LLM accessibility for coding by reducing computational requirements to enable local execution on consumer-grade hardware, and supporting faster inference times critical for real-time development feedback.

Stateful network functions are increasingly used in data centers. However, their scalability remains a significant challenge since parallelizing packet processing across multiple cores requires careful configuration t o avoid compromising the application's semantics or performance. This challenge is particularly important when deploying multiple stateful functions on multi-core servers. This paper proposes FlowMage, a system that leverages Large Language Models (LLMs) to perform code analysis and extract essential information from stateful network functions (NFs) prior to their deployment on a server. FlowMage uses this data to find an efficient configuration of an NF chain that maximizes performance while preserving the semantics of the NF chain. Our evaluation shows that, utilizing GPT-4, FlowMage is able to find and apply optimized configuration when deploying stateful NFs chain on a server, resulting in significant p erformance improvement (up to 11×) in comparison to the default configuration of the system.

GPUs are emerging as the most popular accelerator for many applications, powering the core of machine learning applications. In networked GPU-accelerated applications input & output data typically traverse the CPU and the OS network stack multiple times, getting copied across the system's main memory. These transfers increase application latency and require expensive CPU cycles, reducing the system's efficiency, and increasing the overall response times. These inefficiencies become of greater importance in latency-bounded deployments, or with high throughput, where copy times could quickly inflate the response time of modern GPUs. We leverage the efficiency and kernel-bypass benefits of RDMA to transfer data in and out of GPUs without using any CPU cycles or synchronization. We demonstrate the ability of modern GPUs to saturate a 100-Gbps link, and evaluate the network processing time in the context of an inference serving application.

Background/Aim. There is limited data on the frequency of intracranial hemorrhage (ICrH) in the first wave [beta variant of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)] and second wave (delta variant of SARS-CoV-2) coronavirus disease 2019 (COVID-19) pandemic. The aim of this study was to analyze the appearance of ICrH in COVID-19 patients (CP). Methods. Among 505 CP treated at the Special Hospital for Cerebrovascular Diseases ?Sveti Sava? intermittently during the 2020-2021 period, ICrH was diagnosed in 76 (15.1%) patients. The available information from the medical records regarding clinical, demographic, as well as radiological data (multislice computed tomography examination of the endocranium) was collected and analyzed. Results. In the first wave, out of 308 CP, 63 (20.5%) were diagnosed with ICrH. In the second wave, out of 197 CP, ICrH was diagnosed in 13 (6.6%) patients, which was a statistically significant difference (p < 0.002). There was no statistically significant difference for the presence of hyper-tension (p = 0.271), diabetes mellitus (p = 0.558), and chronic obstructive pulmonary disease (p = 0.794) among CP with ICrH comparing the two waves of the pandemic. However, a statistically significant difference was proven in the frequency of patients with atrial fibrillation and anticoagulant drug therapy (p = 0.021 each). There was no statistically significant difference in the frequency of patients with fever (p = 0.637), fatigue (p = 0.587), hemiparesis (p = 0.831), respiratory symptoms (p = 0.289), and loss of consciousness (p = 0.247). D-dimer values in the second pandemic wave were statistically significantly lower (p = 0.003). The combina-tion of ischemic stroke and ICrH was six times more common in the second wave (p = 0.003). However, cer-ebral parenchymal hemorrhage was two times less frequent (p = 0.001) in the second wave but with statisti-cally higher frequencies of multifocal (23.1%) and dif-fuse type (36.4%) of ICrH (p = 0.007). Combined hem-orrhages, as well as subarachnoid and subdural subtypes, were more common in the second wave (23.1% each). Fatal outcomes occurred in 38.1% of patients in the first wave compared to 69.2% in the second wave (p = 0.039). Conclusion. In the first pandemic wave of COVID-19, ICrH in CP was significantly more frequent and D-dimer was singled out in laboratory analyses, the values of which were statistically significantly higher in comparison with second wave. In the second wave of COVID-19, parenchymal ICrH was less frequent and multifocal and diffuse ICrH were more common in CP with ICrH. The mortality rate was very high in the sec-ond wave.

Data centers increasingly utilize commodity servers to deploy low-latency Network Functions (NFs). However, the emergence of multi-hundred-gigabit-per-second network interface cards (NICs) has drastically increased the performance expected from commodity servers. Additionally, recently introduced systems that store packet payloads in temporary off-CPU locations (e.g., programmable switches, NICs, and RDMA servers) further increase the load on NF servers, making packet processing even more challenging. This paper demonstrates existing bottlenecks and challenges of state-of-the-art stateful packet processing frameworks and proposes a system, called FAJITA, to tackle these challenges & accelerate stateful packet processing on commodity hardware. FAJITA proposes an optimized processing pipeline for stateful network functions to minimize memory accesses and overcome the overheads of accessing shared data structures while ensuring efficient batch processing at every stage of the pipeline. Furthermore, FAJITA provides a performant architecture to deploy high-performance network functions service chains containing stateful elements with different state granularities. FAJITA improves the throughput and latency of high-speed stateful network functions by ~2.43x compared to the most performant state-of-the-art solutions, enabling commodity hardware to process up to ~178 Million 64-B packets per second (pps) using 16 cores.

Could information about future incoming packets be used to build more efficient CPU-based packet processors? Can such information be obtained accurately? This paper studies novel packet processing architectures that receive external hints about which packets are soon to arrive, thus enabling prefetching into fast cache memories of the state needed to process them, just-in-time for the packets' arrival. We explore possible approaches to (i) obtain such hints either from network devices or the end hosts in the communication and (ii) use these hints to better utilize cache memories. We show that such information (if accurate) can improve packet processing throughput by at least 50%.

Battery-free sensor tags are devices that leverage backscatter techniques to communicate with standard IoT devices, thereby augmenting a network's sensing capabilities in a scalable way. For communicating, a sensor tag relies on an unmodulated carrier provided by a neighboring IoT device, with a schedule coordinating this provisioning across the network. Carrier scheduling–computing schedules to interrogate all sensor tags while minimizing energy, spectrum utilization, and latency–is an NP-Hard optimization problem. Recent work introduces learning-based schedulers that achieve resource savings over a carefully-crafted heuristic, generalizing to networks of up to 60 nodes. However, we find that their advantage diminishes in networks with hundreds of nodes, and degrades further in larger setups. This paper introduces RobustGANTT, a GNN-based scheduler that improves generalization (without re-training) to networks up to 1000 nodes (100x training topology sizes). RobustGANTT not only achieves better and more consistent generalization, but also computes schedules requiring up to 2x less resources than existing systems. Our scheduler exhibits average runtimes of hundreds of milliseconds, allowing it to react fast to changing network conditions. Our work not only improves resource utilization in large-scale backscatter networks, but also offers valuable insights in learning-based scheduling.

Detecting anomalies in electrocardiogram data is crucial to identifying deviations from normal heartbeat patterns and providing timely intervention to at-risk patients. Various AutoEncoder models (AE) have been proposed to tackle the anomaly detection task with ML. However, these models do not consider the specific patterns of ECG leads and are unexplainable black boxes. In contrast, we replace the decoding part of the AE with a reconstruction head (namely, FMM-Head) based on prior knowledge of the ECG shape. Our model consistently achieves higher anomaly detection capabilities than state-of-the-art models, up to 0.31 increase in area under the ROC curve (AUROC), with as little as half the original model size and explainable extracted features. The processing time of our model is four orders of magnitude lower than solving an optimization problem to obtain the same parameters, thus making it suitable for real-time ECG parameters extraction and anomaly detection.

Novel backscatter communication techniques enable battery-free sensor tags to interoperate with unmodified standard IoT devices, extending a sensor network’s capabilities in a scalable manner. Without requiring additional dedicated infrastructure, the battery-free tags harvest energy from the environment, while the IoT devices provide them with the unmodulated carrier they need to communicate. A schedule coordinates the provision of carriers for the communications of battery-free devices with IoT nodes. Optimal carrier scheduling is an NP-hard problem that limits the scalability of network deployments. Thus, existing solutions waste energy and other valuable resources by scheduling the carriers suboptimally. We present DeepGANTT, a deep learning scheduler that leverages graph neural networks to efficiently provide near-optimal carrier scheduling. We train our scheduler with optimal schedules of relatively small networks obtained from a constraint optimization solver, achieving a performance within 3% of the optimum. Without the need to retrain, our scheduler generalizes to networks 6 × larger in the number of nodes and 10 × larger in the number of tags than those used for training. DeepGANTT breaks the scalability limitations of the optimal scheduler and reduces carrier utilization by up to compared to the state-of-the-art heuristic. As a consequence, our scheduler efficiently reduces energy and spectrum utilization in backscatter networks.

High-speed ASIC switches hold great promise for offloading complex packet processing pipelines directly in the high-speed data-plane. Yet, a large variety of today's packet processing pipelines, including stateful network functions and packet schedulers, require storing some (or all the) packets for short amount of times in a programmatic manner. Such a programmable buffer feature is missing on today's high-speed ASIC switches. In this work, we present RIBOSOME, a system that extends programmable switches with external memory (to store packets) and external general-purpose packet processing devices such as CPUs or FPGAs (to perform stateful operations). As today's packet processing devices are bottlenecked by their network interface speeds, RIBOSOME carefully transmits only the relevant bits to these devices. RIBOSOME leverages spare bandwidth from any directly connected servers to store the incoming payloads through RDMA. Our evaluation shows that RIBOSOME can process 300G of traffic through a stateful packet processing pipeline (e.g., firewall, load balancer, packet scheduler) by running the pipeline logic on a single server equipped with one 100G interface.

Background/Aim. Since 2014 and the publication of the results of the first study on the accumulation of gadolinium contrast, we have witnessed a growing body of evidence on the deposition and retention of gadolinium in the brain after the use of gadolinium-based contrast agents (GBCAs). However, there is still no strong clinical evidence of the adverse effects of GBCAs on the brain parenchyma. The aim of the study was to determine the existence of gadolinium deposits in the brain of patients with relapsing-remitting multiple sclerosis after a ten-year follow-up period. During this period, the patients have regularly, each year, undergone magnetic resonance imaging (MRI) with the administration of gadolinium contrast (gadopentetate dimeglumine ? Magnevist?) in order to follow the course of the disease. Methods. A cohort of 20 patients was formed for the purpose of this study. The ratio of the values of the signal intensity (SI) of different regions of the brain-to-cerebrospinal fluid (CSF) was com-pared for each patient on the initial MRI examination and the MRI examination ten years later. Results. Frontal cortex-to-CSF (p < 0.01), occipital cortex-to-CSF (p < 0.01), the white matter of the corona radiata-to-CSF (p < 0.01), parietal cortex-to-CSF (p < 0.05), thalamus-to-CSF (p = 0.051), putamen-to-CSF (p = 0.06), and anterior and posterior limb of the capsula interna-to-CSF (p = 0.062) SI ratios increased after multiple gadopentetate administrations. An increase in the absolute values of the T1- weighted (T1W) signal in three-quarters of patients was registered in the frontal and occipital cortex and cerebellar hemispheres. A slightly smaller increase in SI, but still greater than 55?65%, was registered in structures of the parietal cortex, putamen, cornu anterior and posterior of the capsula interna, corpus callosum (CC) splenium, pons, thalamus, nucleus caudatus, substantia nigra, CC genu, and temporal cortex. Conclusion. In the cohort of 20 patients, there was a statistically significant increase in SI in the pre-contrast T1W sequence in the following structures: frontal, parietal, and occipital cortex, as well as supratentorial white matter. This result speaks in favor of the existence of chronic accumulation of gadolinium contrast agent gadopentetate dimeglumine in brain structures.

This article explores opportunities to mitigate the performance impact of IOMMU on high-speed network traffic, as used in the Linux kernel. We first characterize IOTLB behavior and its effects on recent Intel Xeon Scalable & AMD EPYC processors at 200 Gbps, by analyzing the impact of different factors contributing to IOTLB misses and causing throughput drop (up to 20% compared to the no-IOMMU case in our experiments). Secondly, we discuss and analyze possible mitigations, including proposals and evaluation of a practical hugepage-aware memory allocator for the network device drivers to employ hugepage IOTLB entries in the Linux kernel. Our evaluation shows that using hugepage-backed buffers can completely recover the throughput drop introduced by IOMMU. Moreover, we formulate a set of guidelines that enable network developers to tune their systems to avoid the "IOTLB wall", i.e., the point where excessive IOTLB misses cause throughput drop. Our takeaways signify the importance of having a call to arms to rethink Linux-based I/O management at higher data rates.