QoS management on S-NUCA many-core processors is challenging due to the physically distributed last-level caches, which introduce core-dependent access latencies and performance heterogeneity. QoS management is further complicated by synchronizations in parallel applications that couple thread progress, making independent per-core DVFS decisions ineffective and thread-level proxies (e.g., IPS/IPC) unreliable indicators of delivered service. This paper proposes a novel heartbeat-driven QoS scheduler that uses application heartbeats as explicit progress feedback and quantifies cross-thread progress imbalance using the coefficient of variation of per-thread interheartbeat intervals. At each scheduling epoch, the controller computes (i) a differential “rate harmonization” DVFS action that reduces progress dispersion across threads and (ii) a uniform QoS DVFS action that regulates the global heartbeat rate to a minimum target. We evaluate our approach using the HotSniper many-core simulator. Results show that our harmonization-based controller meets the QoS target more robustly and reduces average energy consumption by 43.6% relative to comparable baselines, demonstrating that synchronization-aware progress harmonization is useful for energy-efficient QoS on S-NUCA many-cores.

Large Foundation Model (LFM) inference is both memory- and compute-intensive, traditionally relying on GPUs. However, the limited availability and high cost have motivated the adoption of high-performance general-purpose CPUs, especially emerging 3D-stacked Static Non-Uniform Cache Architecture (3D S-NUCA) systems. These architectures offer enhanced bandwidth and locality but suffer from severe thermal challenges and uneven cache latencies due to 3D Networks-on-Chip (NoC). Optimal management of thread migration and V/f scaling is non-trivial due to LFM kernel diversity and system heterogeneity. Existing thermal management approaches often rely on oversimplified analytical models and lack adaptability. We propose AILFM, an Active Imitation Learning (AIL)-based scheduling framework that learns near-optimal thermal-aware scheduling policies from Oracle demonstrations with minimal run-time overhead. AILFM accounts for both core-level performance heterogeneity and kernel-specific behavior in LFMs to maintain thermal safety while maximizing performance. Extensive experiments show that AILFM outperforms state-of-the-art baselines and generalizes well across diverse LFM workloads.

Deep learning (DL) compilers rely on cost models and auto-tuning to optimize tensor programs for target hardware. However, existing approaches depend on large offline datasets, incurring high collection costs and offering suboptimal transferability across platforms. In this paper, we introduce TCL, a novel efficient and transferable compiler framework for fast tensor program optimization across diverse hardware platforms to address these challenges. Specifically, TCL is built on three core enablers: (1) the RDU Sampler, a data-efficient active learning strategy that selects only 10

Multimodal spatial reasoning often relies on long chains of intermediate textual and visual thoughts, where accumulating visual tokens and dense cross-modal attention incur substantial computation and memory overhead. To address this challenge, we propose Spectral-Progressive Thought Flow (SpecFlow), a novel lightweight multimodal spatial reasoning framework that represents intermediate visual thoughts in a fixed-size discrete cosine space. By exploiting strong energy compaction, SpecFlow preserves global layout and relational structure while introducing high-frequency details only when increased spatial precision is required. To align visual state evolution with linguistic intent, classifier-free guidance enables autoregressive textual thoughts to steer flow-based updates of the visual workspace/state without expanding the context. As a result, SpecFlow maintains a bounded visual workspace whose updates depend only on the current visual state and accumulated textual trace, enabling long-horizon inference with stable latency and memory usage independent of reasoning depth. Empirical results show that SpecFlow achieves competitive or superior reasoning performance while reducing computation and KV cache costs by up to 2.1 times.

Transformer-based models such as BERT have achieved remarkable success across a wide range of natural language processing tasks. However, deploying them on resource-constrained devices, including System-on-Chips (SoCs), remains challenging due to limited memory and computational capacity of these devices. Weight sharing has proved to be an effective approach for significantly reducing model size, yet designing an efficient layer-wise weight sharing framework that achieves high compression while maintaining model performance and low latency is not a trivial task. In this paper, we systematically analyze the impact of key weight sharing hyperparameters on model performance, model size, and latency, and propose a Genetic Algorithm-based optimization framework to automatically identify the optimal configuration for each layer—balancing performance, compression ratio, and inference speed. Additionally, through targeted modifications and efficient on-the-fly decompression, we demonstrate that the proposed weight sharing framework introduces no additional latency overhead and can even reduce inference time. Compared to uniform compression, our approach makes transformer blocks with substantially lower memory requirements and reduced latency while maintaining competitive performance across evaluation metrics.

Deep learning (DL) compilers enable automated deployment of deep neural networks (DNNs) across diverse hardware platforms by leveraging cost models for efficient tensor program optimization. The effectiveness of these systems depends on the prediction accuracy and cross-model generalization capability of offlinetrained cost models. Existing approaches mainly focus on feature engineering and cost model architecture design, while largely overlooking structural similarity at the computational subgraph level, which limits further improvements in prediction performance. To address this limitation, we propose an unsupervised subgraph clustering framework based on graph autoencoders to enhance offline-trained cost models. Specifically, DL models are partitioned into computational subgraphs, and node features are extracted. A graph autoencoder learns high-dimensional structural embeddings of these subgraphs, followed by K-means clustering to group structurally similar patterns. Dedicated cost models are trained for different subgraph clusters to predict tensor program performance. This fine-grained, cluster-aware training strategy improves both prediction accuracy and generalization capability, thereby enhancing auto-tuning efficiency. Experimental results across multiple DL models demonstrate significant improvements in search efficiency and latency optimization on both CPU and GPU platforms. Compared with Tenset, our method achieves average search speedups of 9.46 & times; and 5.15 & times;, while achieving inference latency improvements of 1.18 & times; and 1.07 & times;.

As semiconductor manufacturing advances from the 3-nm process toward the sub-nanometer regime and transitions from FinFETs to gate-all-around field-effect transistors (GAAFETs), the resulting complexity and manufacturing challenges continue to increase. In this context, 3D chiplet-based approaches have emerged as key enablers to address these limitations while exploiting the expanded design space. Specifically, chiplets help address the lower yields typically associated with large monolithic designs. This paradigm enables the modular design of heterogeneous systems consisting of multiple chiplets (e.g., CPUs, GPUs, memory) fabricated using different technology nodes and processes. Consequently, it offers a capable and cost-effective strategy for designing heterogeneous systems. This paper introduces the Horizon Europe Twinning project COIN-3D (Collaborative Innovation in 3D VLSI Reliability), which aims to strengthen research excellence in 2.5D/3D VLSI systems reliability through collaboration between leading European institutions. More specifically, our primary scientific goal is the provision of novel open-source Electronic Design Automation (EDA) tools for reliability assessment of 3D systems, integrating advanced algorithms for physical- and system-level reliability analysis.

Power Delivery Networks (PDNs) are critical for maintaining voltage integrity in modern multiprocessor systems. Conventional early-stage PDN planning relies on static or worst-case power assumptions, often leading to over-provisioned designs and inefficient use of routing resources. This paper proposes a workload-aware methodology for early-stage PDN optimization based on architectural power traces. Using architectural simulations, temporal power activity is captured at fine granularity and mapped to spatial power density distributions across the chip. These distributions are then translated into current demand profiles to guide PDN topology planning at tile granularity. By incorporating realistic workload behavior, the proposed approach enables adaptive PDN resource allocation during early design stages. Experimental results demonstrate that the method achieves up to 32.94

Transformer models have set new performance standards for machine learning (ML) tasks. However, their resource-intensive deployment on resource-constrained edge devices for cloud-free, on-chip transformer inference remains challenging. The ARM Compute Library (ARM-CL) framework provides low-latency CNN inference on ARM-based edge devices but lacks support for transformer inference. In this work, we implement several new transformer kernels in ARM-CL to support native transformer execution. Our extended ARM-CL achieves up to three times faster transformer inference compared to state-of-the-art CPU/GPU implementations on an ARM-based embedded board. Furthermore, heterogeneous multi-processor system-on-chips (HMPSoCs) powering edge devices provide both embedded CPUs and GPUs. We introduce cooperative CPU-GPU transformer inference, which executes memory-intensive operations on the CPU while utilizing the GPU for highly parallelizable, compute-intensive operations. This cooperative execution, implemented with minimal overhead, further reduces transformer inference latency by up to 15.72

Recently, visualization-of-thought (VoT) has unlocked new opportunities for complex spatial reasoning in multimodal large language models (MLLMs) by complementing verbal reasoning with visual thinking. However, the autoregressive accumulation of lengthy and redundant tokens substantially increases computation and memory costs. In this paper, we present a new efficient framework for multimodal spatial reasoning, named *DARE*, designed to adaptively prune multimodal tokens across different network depths, reasoning hops, and modalities. First, *DARE* devises an intra- and inter-hop-aware differentiable retention mechanism to dynamically estimate token importance both within each reasoning step and across successive hops. Recognizing that deeper network layers encode visual cues into verbal streams, *DARE* introduces an asymmetric compression strategy that prunes tokens according to modality-specific redundancy and semantic importance. Furthermore, *DARE* incorporates a progressive KV-cache retention policy aligned with cross-modal fusion dynamics, further reducing memory overhead during autoregressive reasoning. Our method delivers substantial reductions in computation and memory footprint, averaging a 40.37\% reduction in FLOPs and 46.07\% reduction in KV caches usage, while consistently preserving or even improving reasoning performance across seven multimodal spatial reasoning benchmarks, and further generalizing to broader multimodal reasoning tasks, establishing a scalable and robust recipe for efficient multimodal reasoning.

The energy consumption of digital services has become a concern for stakeholders committed to sustainability. Raising awareness of this consumption is essential to improve the energy efficiency of digital services. However, expressing the energy usage of digital services in an easily understandable and actionable way remains a challenge. We address this challenge by proposing a first operational energy labeling method for digital services in the computing continuum. Our approach enables stakeholders, including cloud and network providers, application developers, researchers, and end-users of digital services, to better understand and improve the energy efficiency of their applications. Focusing on video surveillance digital services, and using the enhanced iFogSim framework, we propose an energy labeling scheme, and demonstrate its merits with extensive scenario analysis and simulation. We further discuss how our approach can help reduce energy consumption and/or improve performance, all without modifying the application's functional parameters or system architecture.

Low-rank adaptation (LoRA) has been demonstrated effective in reducing the trainable parameter number when fine-tuning a large foundation model (LLM). However, it still encounters computational and memory challenges when scaling to larger models or addressing more complex task adaptation. In this work, we introduce Sparse Spectrum Adaptation via Discrete Hartley Transformation (SSH), a novel approach that significantly reduces the number of trainable parameters while enhancing model performance. It selects the most informative spectral components across all layers, under the guidance of the initial weights after a discrete Hartley transformation (DHT). The lightweight inverse DHT then projects the spectrum back into the spatial domain for updates. Extensive experiments across both single-modality tasks such as language understanding and generation and multi-modality tasks such as video-text understanding demonstrate that SSH outperforms existing parameter-efficient fine-tuning (PEFT) methods while achieving substantial reductions in computational cost and memory requirements.

Using Heterogeneous Multi-Processor System-on-Chips (HMPSoCs) for Deep Neural Network (DNN) inference has become commonplace in edge devices. However, reducing the DNN inference latency on resource-constrained edge devices remains a first-class constraint. Early-Exit (EE) DNNs offer variable, input-dependent inference latency, aiming to reduce average inference latency by lowering the latency of simple inputs at the cost of increased latency for complex inputs. However, the overheads introduced by exit branches reduce the potential performance gains with EE DNNs.Furthermore, existing CPU- or GPU-only implementations of EE DNN inference under-utilize the HMPSoC. To address this limitation, we propose a cooperative and parallel CPU-GPU execution approach1 for EE DNN inference that effectively distributes computations across all HMPSoC processors, minimizing latency variations. Our approach allows EE DNNs to achieve reduced average latency on HMPSoCs relative to their static counterparts, significantly reducing average- and worst-case inference latencies and enhancing the speed-up of EE DNN compared to the best single-processor inference. On average, the worst-case inference latency decreased by 24.8% across three commonly used EE DNNs, providing latency comparable to a static model without compromising accuracy on an RK3399PRO HMPSoC.

As edge-based deep learning applications become more complex, optimizing performance on heterogeneous System-on-Chips (SoCs) presents unique challenges. Traditional pipelining techniques distributing the computation across different on-chip processing units, while effective for throughput, do not address the latency demands posed by modern neural networks with complex interdependencies and extensive operator parallelism. There is a potential in leveraging operator parallelism to enable concurrent execution across multiple processing units, thereby reducing inference latency. However, prioritizing pipelining or parallel execution often necessitates a compromise, where optimizing one performance metric adversely impacts the other. This article introduces Para-Pipe, a hierarchical mapping framework that integrates intra- and interstage operator parallelism within a pipelined architecture. Para-Pipe navigates the tradeoff between throughput and latency by selectively fine-tuning parallelism levels within and across pipeline stages. This strategy can significantly reduce interprocessor communication overhead, significantly improving energy efficiency. Our evaluation demonstrates that Para-Pipe generates multiple Pareto-optimal configurations, achieving a balance between throughput and latency on an Amlogic SoC equipped with ARM big.LITTLE CPUs and GPU, as well as the Black Sesame Technology SoC featuring a deep learning accelerator and two DSPs. More importantly, throughput-optimized configurations under Para-Pipe on Amlogic SoC show an average energy efficiency improvement of 11.0% over purely pipelined strategies and 23.3% relative to nonpipelined parallel execution.

Optimizing performance and energy efficiency in many-core processors, especially within Non-Uniform Cache Access (NUCA) architectures, remains a critical challenge. The performance heterogeneity inherent in S-NUCA systems complicates task scheduling due to varying cache access latencies across cores. This paper introduces a novel QoS management policy to maintain application execution within predefined Quality of Service (QoS) targets, measured using the Application Heartbeats framework. QoS metrics like Heartbeats ensure predictable application performance in dynamic computing environments. The proposed policy dynamically controls QoS by orchestrating task migrations within the S-NUCA many-core system and adjusting the clock frequency of cores. After satisfying the QoS objectives, the policy optimizes energy efficiency, reducing overall system energy consumption without compromising performance constraints. Our work leverages the state-of-the-art multi-/many-core simulator HotSniper. We have extended it with two key components: an integrated heartbeat framework for precise, application-specific performance monitoring, and our QoS management policy that maintains application QoS requirements while minimizing the system's energy consumption. Experimental evaluations demonstrate that our approach effectively maintains desired QoS levels and achieves 18.7% energy savings compared to state-of-the-art scheduling methods.

We present a new adaptation method MaCP, Minimal yet Mighty adaptive Cosine Projection, that achieves exceptional performance while requiring minimal parameters and memory for fine-tuning large foundation models. Its general idea is to exploit the superior energy compaction and decorrelation properties of cosine projection to improve both model efficiency and accuracy. Specifically, it projects the weight change from the low-rank adaptation into the discrete cosine space. Then, the weight change is partitioned over different levels of the discrete cosine spectrum, and each partition's most critical frequency components are selected. Extensive experiments demonstrate the effectiveness of MaCP across a wide range of single-modality tasks, including natural language understanding, natural language generation, text summarization, as well as multi-modality tasks such as image classification and video understanding. MaCP consistently delivers superior accuracy, significantly reduced computational complexity, and lower memory requirements compared to existing alternatives.

Power Delivery Network (PDN) design is a wellknown challenge, especially for multiprocessor systems. The PDN design process must account for increasing power density, stringent voltage noise requirements, and complex current distribution across multiple cores. In this paper, we propose a die-centric, system-level aware PDN design methodology that can be applied to systems with a large number of cores and significantly reduces design cycle time. Our methodology leverages architectural power consumption values as well as the floorplan of the design to generate accurate and electromigration (EM)-safe PDNs for largescale multiprocessor systems. Experimental results demonstrate that our methodology can generate PDNs with minimal IR drop while maintaining EM stress levels well below critical thresholds.

HMPSoCs combine different processors on a single chip. They enable powerful embedded devices, which increasingly perform ML inference tasks at the edge. State-of-the-art HMPSoCs can perform on-chip embedded inference using different processors, such as CPUs, GPUs, and NPUs. HMPSoCs can potentially overcome the limitation of low single-processor CNN inference performance and efficiency by cooperative use of multiple processors. However, standard inference frameworks for edge devices typically utilize only a single processor. We present the ARM-CO-UP framework built on the ARM-CL library. The ARM-CO-UP framework supports two modes of operation – Pipeline and Switch. It optimizes inference throughput using pipelined execution of network partitions for consecutive input frames in the Pipeline mode. It improves inference latency through layer-switched inference for a single input frame in the Switch mode. Furthermore, it supports layer-wise CPU/GPU DVFS in both modes for improving power efficiency and energy consumption. ARM-CO-UP is a comprehensive framework for multi-processor CNN inference that automates CNN partitioning and mapping, pipeline synchronization, processor type switching, layer-wise DVFS , and closed-source NPU integration.

In today’s digital society, diverse computing de-vices-from edge devices to data centers-support various applications, each with specific performance and energy characteristics. Analyzing application energy consumption is crucial for improving energy efficiency, optimizing resources, and reducing environmental impact. However, while comprehensive energy measurements are feasible for specific configurations, they are impractical for assessing diverse application mappings. Still, stakeholders such as cloud providers, developers, users, and researchers need insights into application-level energy behavior for informed decision-making. In this work, we propose a fine-grained simulation approach for analyzing application energy behavior in edge/cloud environments. We implemented our approach as an enhanced version of the iFogSim framework. We demonstrate its effectiveness by evaluating different multi-application scenarios and configurations for a video surveillance application. Our approach facilitates the fast evaluation of different scenarios and deployment strategies, providing insights that can contribute to more energy-efficient edge/cloud computing systems and digital services.

Deep Neural Network (DNN) inference is now ubiquitous in embedded applications at the edge. State-of-the-art Heterogeneous Multi-Processors System-on-Chip (HMPSoCs) powering these applications come equipped with powerful Neural Processing Units (NPUs) that significantly outperform other inference-capable HMPSoC components - namely, the CPUs and GPUs - in terms of power consumption and performance. However, CPUs and GPUs can perform full precision inference, whereas NPUs can often only perform a quantized inference. Consequently, low-latency, low-power inference by the NPU comes at an accuracy loss due to the quantization. DNNs consist of several heterogeneous layers. Here, we introduce the PiQi framework that allows DNN inference to layer-wise switch between the three inference-capable HMPSoC components, CPU, GPU, and NPU, mid-inference with minimal overhead. Consequently, PiQi employs the novel idea of partially quantized DNN inference on HMPSoCs. However, different DNN layers experience different power-performance gains while projecting different accuracy losses on quantization. Therefore, we provide within PiQi a multi-objective Genetic Algorithm (GA) that provides a power-performance Pareto-front under an accuracy constraint by selective multi-layer quantization during inference. Additionally, PiQi utilizes a neural network to expedite search time by predicting accuracy when assigning DNN layers to the appropriate cores.