LLM deployment on resource-constrained edge devices faces severe latency constraints, particularly in real-time applications where delayed responses can compromise safety or usability. Among many approaches to mitigate the inefficiencies of sequential token-by-token generation, Speculative Decoding (SD) has emerged as a promising technique. However, SD at the edge is hindered by two major challenges: (1) integrating SD into a compiler-based workflow without sacrificing performance or programmability, and (2) exploiting the heterogeneous compute resources of modern SoCs through carefully designed partitioning strategies. This work addresses these challenges by using an analytical cost model that explores heterogeneous hardware configurations and guides coarse-grained partitioning of LLM subgraphs, particularly with edge-typical short input sequence lengths. The cost model predicts when speculative sampling and heterogeneous execution are jointly beneficial and is validated on an edge device featuring a hexacore Cortex-A CPU and a Mali GPU, revealing up to 1.68× speedup for translation tasks, closely matching analytic expectations.

Compilers for general-purpose languages have been shown to be at a disadvantage when it comes to specialized application domains as opposed to their Domain-Specific Language (DSL) counterparts. However, the field of DSL compilers features little consolidation in terms of compiler frameworks and adjacent software ecosystems. As a result, considerable work is duplicated, lost to maintenance issues, or remains undiscovered, and most DSLs are never considered "production-ready". One notable development is the introduction of the Multi-Level Intermediate Representation (MLIR), which promises a similar impact on DSL compilers as LLVM had on general-purpose tooling. In this work, we present a NumPy-like DSL made for offloading numeric tensor kernels that is entirely MLIR-native. In a first for open-source, it implements all frontend actions and semantic analyses directly within MLIR. Most notably, this is made possible by our new dialect-agnostic MLIR type checker, created for the future of DSLs in MLIR. We implement a simple, yet effective, parallel-first lowering scheme that connects our language to another MLIR dataflow dialect for seamless offloading. We show that our approach performs well in real-world use cases from the domain of weather modeling and Computational Fluid Dynamics (CFD) in Fortran.

Computing-in-memory (CIM) has been demonstrated across various memory technologies, ranging from memristive crossbars performing analog dot-product computations to large-scale digital bitwise operations in commodity DRAM and other proposed non-volative memory technologies. However, current CIM solutions face latency and reliability challenges. CIM fidelity lags considerably behind access fidelity. Furthermore, bulk-bitwise CIM, although highly parallelized, requires long latency for operations like multiplication and addition, due to their bit-serial computation. This paper presents Count2Multiply, a technology-agnostic digital CIM approach to perform multiplication, addition and other operations using high-radix, massively parallel counting enabled by CIM bulk-bitwise logic operations. Designed to meet fault tolerance requirements, Count2Multiply integrates traditional row-wise error correction codes, such as Hamming and BCH, to address the high error rates in existing CIM designs. We demonstrate Count2Multiply with a detailed application to CIM in conventional DRAM due to its ubiquity and high endurance. However, we note that the Count2Multiply architecture is compatible with other functionally complete CIM proposals. Compared to the state-of-the-art in-DRAM CIM method, Count2Multiply achieves up to 10x speedup, 8x higher GOPS/Watt, and 9.5x higher GOPS/area, while outperforming GPU for vector-matrix multiplications.

The adoption of high-performance multi-core platforms in avionics and automotive systems introduces significant challenges in ensuring predictable execution, primarily due to shared resource interferences. Many existing approaches study interference from a single angle-for example, through hardware-level analysis or by monitoring software execution. However, no single abstraction level is sufficient on its own. Hardware behavior, program structure, and system configuration all interact, and a complete view is needed to understand where interferences come from and how to reduce them. In this paper, we present a methodology that brings together several tools that operate at different abstraction levels. At the lowest level, PHYLOG provides a formal model of the hardware and identifies possible interference channels using micro-architectural transactions. At the program level, machine learning analysis locates the exact parts of the code that are most sensitive to shared-resource contention. At the compilation level, MLIR-based transformations use this information to reshape memory access patterns and reduce pressure on shared resources. Finally, at the system level, Linux cgroups enforce static execution constraints to prevent highly interfering tasks from running together. The goal of our approach is to reduce memory interference and improve the system's predictability, thereby easing the certification process of multi-core systems in safety-critical domains.

Driven by the increasing demand for low-latency and real-time processing, machine learning applications are steadily migrating toward edge computing platforms, where Field-Programmable Gate Arrays (FPGAs) are widely adopted for their energy efficiency compared to CPUs and GPUs. To generate high-performance and low-power FPGA designs, several frameworks built upon High Level Synthesis (HLS) vendor tools have been proposed, among which MLIR-based frameworks are gaining significant traction due to their extensibility and ease of use. However, existing state-of-the-art frameworks often overlook the stringent resource constraints of edge devices. To address this limitation, we propose MING, an Multi-Level Intermediate Representation (MLIR)-based framework that abstracts and automates the HLS design process. Within this framework, we adopt a streaming architecture with carefully managed buffers, specifically designed to handle resource constraints while ensuring low-latency. In comparison with recent frameworks, our approach achieves on average 15x speedup for standard Convolutional Neural Network (CNN) kernels with up to four layers, and up to 200x for single-layer kernels. For kernels with larger input sizes, MING is capable of generating efficient designs that respect hardware resource constraints, whereas state-of-the-art frameworks struggle to meet.

The deployment of convolutional neural network (CNN) in real-time and resource-constrained applications poses critical challenges due to their computational demands and the need for reliable decision making. In this article, we combine adaptive inference via early exits (EE) with selective prediction (SP) to address these challenges. Early exits allow confident predictions at intermediate layers, while SP introduces uncertainty estimation modules, enabling the system to abstain from low-confidence decisions or continue inference through deeper layers. This combined design lowers the risk of overconfident but erroneous predictions and improves the tradeoff between performance and accuracy. As a case study, we implement and evaluate our approach on a real-time traffic sign detection task, processing the input of an RGB camera in the forward direction. In this article, we demonstrate improved performance compared to baseline models. Compared to SP-only (SP) and early exiting (EE)-only (Early Exit) baselines, our hybrid model achieves low inference depth (1.20), leading to reduced computational demand and latency. Despite this efficiency, the model maintains a high prediction accuracy (90.3%) and a low abstention rate (1.6%), ensuring fast and reliable decision making suitable for time-critical embedded applications. This demonstrates an effective tradeoff between effective computation and predictive reliability.

Data transfer between processors and memory remains a critical bottleneck in improving application performance on traditional computing hardware, particularly for data-intensive workloads such as machine learning, bioinformatics, and security applications. In-memory computing (IMC), a paradigm where a substantial portion of data processing occurs directly within memory, has emerged as a promising solution to mitigate this bottleneck. The advancement of emerging non-volatile memory (NVM) technologies has further accelerated the development of IMC hardware fabrics. However, harnessing the full potential of IMC requires a cross-layer design approach that spans memory technologies, circuits, architectures, and systems. Essential cross-layer tools—including modeling and simulation, data partitioning and mapping, and operation scheduling—play a pivotal role in designing efficient IMC-based hardware. This article reviews key advancements in simulation and design tools for IMC fabrics, with a focus on NVM-based crossbar arrays and content-addressable memories, while highlighting the necessity of cross-layer collaboration. Additionally, we discuss current challenges and emerging opportunities in the field.

Hyperdimensional computing (HDC) is an emerging brain-inspired machine learning framework built upon unique properties of high-dimensional vectors. The vectors can contain floating-point (FP) or binary values, offering tradeoffs in terms of accuracy and computational cost. Previous works have proposed quantization methods to convert FP models into binary ones to improve performance. Unfortunately, these approaches not only incur an accuracy loss but also sacrifice valuable properties of HDC, such as low training time or robustness to noise. To overcome these limitations, we propose TQHD, a quantization method that transforms FP vectors into thermometer-encoded binary vectors. TQHD reduces the accuracy loss inflicted by quantization by 3.4 pp in complex scenarios compared to the state-of-the-art.

As the demand for efficient, low-power computing in embedded and edge devices grows, traditional computing methods are becoming less effective for handling complex tasks. Stochastic computing (SC) offers a promising alternative by approximating complex arithmetic operations, such as addition and multiplication, using simple bitwise operations, like majority or AND, on random bit-streams. While SC operations are inherently fault-tolerant, their accuracy largely depends on the length and quality of the stochastic bit-streams (SBS). These bit-streams are typically generated by CMOS-based stochastic bit-stream generators that consume over 80% of the SC system's power and area. Current SC solutions focus on optimizing the logic gates but often neglect the high cost of moving the bit-streams between memory and processor. This work leverages the physics of emerging ReRAM devices to implement the entire SC flow in place: (1) generating low-cost true random numbers and SBSs, (2) conducting SC operations, and (3) converting SBSs back to binary. Considering the low reliability of ReRAM cells, we demonstrate how SC's robustness to errors copes with ReRAM's variability. Our evaluation shows significant improvements in throughput (1.39x, 2.16x) and energy consumption (1.15x, 2.8x) over state-of-the-art (CMOS- and ReRAM-based) solutions, respectively, with an average image quality drop of 5% across multiple SBS lengths and image processing tasks.

To design performant, expressive, and reliable cyber-physical systems (CPSs), researchers extensively perform quasi-static scheduling for concurrent models of computation (MoCs) on multi-core hardware. However, these quasi-static scheduling approaches are developed independently for their corresponding MoCs, despite commonality in the approaches. To help generalize the use of quasi-static scheduling to new and emerging MoCs, this article proposes a unified approach for a class of deterministic timed concurrent models (DTCMs), including prominent models such as synchronous dataflow (SDF), Boolean-controlled dataflow (BDF), scenario-aware dataflow (SADF), and Logical Execution Time (LET). In contrast to scheduling techniques tailored exclusively to specific MoCs, our unified approach leverages a common intermediate formalism called state space finite automata (SSFA), bridging the gap between high-level MoCs and executable schedules. Once identified as DTCMs, new MoCs can directly adopt SSFA-based scheduling, significantly easing adoption. We show that quasi-static schedules facilitated by SSFA are provably free from timing anomalies and enable straightforward worst-case makespan analysis. We demonstrate the approach using the reactor model—an emerging discrete-event MoC—programmed using the Lingua Franca (LF) language. Experiments show that quasi-statically scheduled LF programs exhibit lower runtime overhead compared to the dynamically scheduled LF programs, and that the analyzable worst-case makespans enable compile-time deadline checking.

Reactive software poses challenging requirements: deterministic execution with stringent timing constraints under a tight energy budget. Meeting these requirements is particularly hard when executing on the increasingly heterogeneous platforms of today. In this letter, we integrate Mocasin, a design space exploration tool, into Lingua Franca, a programming framework for building deterministic and timed reactive software. We show that this integration enables choosing a desired timing and energy performance at design time. We demonstrate our approach in satellite attitude control, consisting of periodic real-time tasks and sporadic non-real-time tasks. The latter sporadic tasks are coordinated using quasi-static schedules, computed by Mocasin, leading to less energy consumption compared to the Linux scheduler under CPU frequency scaling governors, such as powersave, schedutil, and ondemand.

Non-volatile Memory (NVM) technologies present a promising alternative to traditional volatile memories such as SRAM and DRAM. Due to the limited availability of real NVM devices, simulators play a crucial role in architectural exploration and hardware-software co-design. This tutorial presents a simulation toolchain through four detailed case studies, showcasing its applicability to various domains of system design, including hybrid main-memory and cache, compute-in-memory, and wear-leveling design. These case studies provide the reader with practical insights on customizing the toolchain for their specific research needs. The source code is open-sourced.

Structured sparsity enables deploying large language models (LLMs) on resource-constrained systems. Approaches like dense-to-sparse fine-tuning are particularly compelling, achieving remarkable structured sparsity by reducing the model size by over 6.7 Chi, while still maintaining acceptable accuracy. Despite this reduction, LLM inference, especially the decode stage being inherently memory-bound, is extremely expensive on conventional Von-Neumann architectures. Compute-in-memory (CIM) architectures mitigate this by performing computations directly in memory, and when paired with sparse LLMs, enable storing and computing the entire model in memory - eliminating the data movement on the off-chip bus and improving efficiency. Nonetheless, naively mapping sparse matrices onto CIM arrays leads to poor array utilization and diminished computational efficiency. In this paper, we present an automated framework with novel mapping and scheduling strategies to accelerate sparse LLM inference on CIM accelerators. By exploiting block-diagonal sparsity, our approach improves CIM array utilization by over 50%, achieving more than 4 Chi reduction in both memory footprint and the number of required floating-point operations.

This brief presents an overview of recent tools and research efforts aimed at enhancing the programmability and reliability of In-Memory Computing (IMC)-based systems. We discuss hardware-aware training techniques that improve model resilience to analog device imperfections, and explore mapping strategies that balance accuracy and performance for heterogeneous IMC-based accelerators. Additionally, we examine a compiler framework that abstracts hardware complexities and enables seamless integration of these accelerators into existing deployment pipelines. By combining these approaches with advanced simulation tools, we propose an end-to-end workflow that facilitates the practical deployment and optimization of IMC technologies across diverse memory types and architectural designs.

Energy efficiency has become a key concern in modern computing. Major processor vendors now offer single-ISA heterogeneous processors that combine powerful and energy-efficient cores, such as Arm's big.LITTLE CPUs, Apple's M-series chips, and Intel P/E systems. However, today's OS schedulers, relying on simple cost-based thread allocation strategies, fail to fully exploit their potential. This paper presents HARP, a Linux-integrated resource-management framework for heterogeneous processors. HARP leverages application behavior through online monitoring or application descriptions and introduces a lightweight interface for two-way communication between applications and the resource manager. Through this interface, HARP learns application characteristics to guide allocation decisions, which are then relayed back to the applications so they can adapt accordingly. HARP supports various programming models, from OpenMP and Intel TBB to custom models with adaptivity features, significantly improving performance and energy efficiency, particularly in multi-application scenarios. On two representative heterogeneous systems, HARP reduces the average execution time by 12 % and the energy consumption by 28 % compared to existing methods. Overall, HARP marks a crucial step toward energy-efficient computing across diverse architectures.

Compute-Near-Memory (CNM) systems offer a promising approach to mitigate the von Neumann bottleneck by bringing computational units closer to data. However, optimizing for these architectures remains challenging due to their unique hardware and programming models. Existing CNM compilers often rely on manual programmer annotations for offloading and optimizations. Automating these decisions by exploring the optimization space, common in CPU/GPU systems, is difficult for CNMs as constructing and navigating the transformation space is tedious and time consuming. This is particularly the case during system-level design, where evaluation requires time-consuming simulations. To address this, we present CoMoNM, a generic cost modeling framework for CNM systems for execution time estimation in milliseconds. It takes a high-level, hardware-agnostic application representation, target system specifications, and a mapping specification as input and estimates the execution time for the given application on the target CNM system. We show how CoMoNM can be seamlessly integrated into state-of-the-art CNM compilers, providing improved offloading decisions. Evaluation on established benchmarks for CNM shows estimation errors within 7.80

The MYRTUS Horizon Europe project embraces the principles of the EU CloudEdgeIoT Initiative, integrating edge, fog, and cloud in a continuum of computing resources. MYRTUS intends to deliver abstractions, cognitive orchestration mechanisms, and a whole design environment to build and operate collaborative, distributed, heterogeneous systems. The goal is to provide high performance and play a crucial role in enabling energy efficiency and trustworthiness in nowadays systems.

Lingua Franca (LF) is a coordination language for cyber-physical systems that ensures deterministic execution through the reactor model. While LF's conservative scheduler guarantees correctness, it can hinder performance due to synchronization barriers that limit parallelism. Timing enclaves, a recent extension to LF, mitigate this by partitioning programs into independently scheduled regions with decoupled logical timelines. However, determining optimal enclave configurations is non-trivial and not intuitive to programmers, as it requires balancing parallelism, synchronization overhead, and system constraints. In this paper, we present a framework for Design Space Exploration (DSE) of timing enclaves in LF programs. Our approach automatically analyzes the program's topology, identifies valid enclave partitions, and uses heuristics to explore trade-offs between determinism and latency. We evaluate the effectiveness of our method on synthetic benchmarks and a realworld 4G baseband processing application. Results show that our DSE framework can significantly improve performance while preserving deterministic semantics, highlighting its practical value for complex real-time systems.

Modern big data workflows are characterized by computationally intensive kernels. The simulated results are often combined with knowledge extracted from AI models to ultimately support decision-making. These energy-hungry workflows are increasingly executed in data centers with energy-efficient hardware accelerators since FPGAs are well-suited for this task due to their inherent parallelism. We present the H2020 project EVEREST, which has developed a system development kit (SDK) to simplify the creation of FPGA-accelerated kernels and manage the execution at runtime through a virtualization environment. This paper describes the main components of the EVEREST SDK and the benefits that can be achieved in our use cases.

The MYRTUS Horizon Europe Project embraces the principles of the EUCloudEdgeIOT Initiative and integrates edge, fog and cloud computing platforms, leveraging a cognitive engine based on swarm intelligence and federated learning to orchestrate collaborative distributed and decentralised components.