Approximate Computing (AxC) is being actively explored to meet the energy and performance requirements of resource-constrained embedded systems. Approximate arithmetic operators (AxOs), for instance, let edge-AI systems trade tiny, bounded errors for big wins in power, performance, and area. This tutorial demystifies AxO design, modeling, and exploration: from platform-aware operator synthesis (e.g., selective LUT pruning) to application-specific DSE that uses AI/ML to navigate massive trade-off spaces. We contrast selection (library) vs. synthesis (generate-and-optimize) flows, show when FPGA-aware adders/multipliers outperform ASIC-ported designs, and connect operator-level error to task-level metrics (e.g., Conv2D, MLP). The tutorial includes hands-on Jupyter notebooks, ready-to-reuse operator models, and a practical recipe for building Pareto-optimal AxOs under accuracy constraints - plus a peek at AxOSyn, an open-source framework that unifies selection/synthesis, surrogate fitness, and search using evolutionary algorithms.
The rapid growth of AI workloads is driving interest in Approximate Computing (AxC) as a means to enable low-cost, energy-efficient inference in resource-constrained systems. By introducing controlled inaccuracies, AxC can deliver substantial gains in power, performance, and area (PPA) while leveraging the inherent error tolerance of many AI models. Achieving this potential requires adapting existing frameworks to support the design and optimization of neural networks with approximate operators. Modern AxC research extends beyond accuracy-PPA trade-offs to address reliability and security, reducing redundancy overheads and exploring the distinctive side-channel implications of approximation. Application-aware approaches, such as those for spiking neural networks, show that tailoring approximation to workload-specific error behavior can surpass generic strategies. This article examines AI-guided design methods and the interplay between efficiency, reliability, and security, highlighting how these interconnected facets can advance embedded and high-performance computing.
This brief presents an overview of recent tools and research efforts aimed at enhancing the programmability and reliability of In-Memory Computing (IMC)-based systems. We discuss hardware-aware training techniques that improve model resilience to analog device imperfections, and explore mapping strategies that balance accuracy and performance for heterogeneous IMC-based accelerators. Additionally, we examine a compiler framework that abstracts hardware complexities and enables seamless integration of these accelerators into existing deployment pipelines. By combining these approaches with advanced simulation tools, we propose an end-to-end workflow that facilitates the practical deployment and optimization of IMC technologies across diverse memory types and architectural designs.
Neural networks have demonstrated superior performance over rule-based and model-based approaches in processing noisy sensing data. However, their substantial computational and energy demands hinder deployment in battery-powered embedded systems. Computing-in-Memory (CIM) devices offer a promising alternative by significantly reducing energy consumption. Prior work [2] achieves this by leveraging a non-von Neumann architecture, which minimizes data movement between memory and compute units, thereby mitigating the memory wall bottleneck. Despite these advantages, analog CIM (ACIM) systems face several key challenges, including analog noise, limited numerical precision, and increased hardware complexity. While cloud-based neural networks are still dominant, emerging applications increasingly demand real-time, privacy-preserving inference on-device. For instance, facial authentication requires local execution on edge devices to ensure low-latency responsiveness and to protect user privacy. CIM architectures are particularly well-suited to these scenarios due to their tightly integrated memory-compute structure, offering low-latency and energy-efficient inference capabilities.
Transformer self-attention offers strong expressiveness, but its compute and memory cost grows rapidly with longer sequences. This results in frequent off-chip memory access, which becomes a major performance bottleneck. FlashAttention reduces this by dividing the sequence into tiles, computed entirely in on-chip memory. This avoids storing intermediate tensors off-chip and alleviates memory bandwidth issues. However, tile-wise online softmax requires floating-point operations for numerical stability using max-based scaling and accumulation. We propose I-FlashAttention, an integeronly version of FlashAttention. It uses shift-based exponential approximation and integer max-tracking to perform online softmax without floating point. All steps, from INT8 GEMM to output, are fused into a single Triton kernel. I-FlashAttention is 1.08x faster than FP16 FlashAttention and 7.10x faster than I-ViT.
This paper proposes RISC-TAE(Transformer Acceleration Engine), a RISC-V instruction set extension with microarchitectural co-design, to meet the performance and energy-efficiency requirements of Transformer models in edge scenes.The design integrates operator-specialized computing units (GEMM/Softmax/GELU) with hardware-managed dataflow orchestration through custom RISC-V instructions, effectively resolving energy efficiency bottlenecks and memory access fragmentation in existing solutions. Experiments demonstrate that RISC-TAE achieves 23.35x and 8.01x speedups over scalar processor CVA6 and vector processor ARA respectively for BERT inference, while outperforming RISC-VTF by 1.4x, providing a scalable solution for edge Transformer deployment.
The rapid advancement of AI (from foundational machine learning to Large Language Models) and edge computing has placed unprecedented demands on computation, memory, and storage on resource-constrained edge devices. As AI models scale, the ability to efficiently manage computing resources, utilize memory and storage, and reduce energy consumption has become critical. This paper introduces contributions on 4 topics related to deploying AI on resource-constrained edge devices: 1) unlocking training of foundational machine learning algorithms on the edge, 2) exploring hardware-aware DNN architecture and mapping co-optimization for inference on heterogeneous systems, 3) scaling RAG by leveraging advanced memory, storage, and energy-efficient designs, and 4) investigating cost-effective and high-performance large-scale graph processing.
In this work, a novel PUF key generation algorithm is introduced that leverages the inherent variability in sensor properties to produce highly entropic keys for security applications. The approach uses variable-length subkeys, which are encoded into a fixed-length final key. The encoding is specifically designed to preserve the subkey length information during key generation. Real-world measurements from high-fidelity inertial MEMS sensors (gyroscopes and accelerometers) were used to verify the proposed technology. Our experimental evaluation demonstrates strong uniqueness and reliability with stable performance under controlled conditions.
Embedded systems are evolving in complexity, leading to the emergence of multiple threats. The co-design and execution of software on the embedded systems further exacerbate the attack surface, making them more vulnerable to sophisticated attacks. As embedded systems are used in critical areas, ensuring their security is crucial. In this special session paper, primarily four major topics regarding embedded systems’ security are discussed. Firstly, this paper initially explores timing channel analysis at a microarchitectural level in heterogeneous hardware to address the security challenges. It then delves into exploring software-based fuzzing techniques to detect vulnerabilities and enhance embedded system security. Additionally, the paper discusses strategies for improving security in IoT devices with a layered defense strategy known as Snowflake IoT. Finally, it examines approaches to securing large and complex monolithic systems. The challenges and opportunities for securing the embedded systems according to the scale and type of attacks.
Transformers have achieved superhuman capabilities in language generation. This comes at a significant energy cost, with inference for models like OpenAI’s ChatGPT estimated to run into hundreds of thousands of dollars per day. In stark contrast, the human brain operates with far great energy efficiency. This extended abstract, based on our educational session at Embedded Systems Week, explores the parallels between spiking neural networks (SNNs) and language models, higlighting their similarities with linear recurrent and state-space models. By examining these connections, we explore a pathway toward the development of energy-efficient language models, which may substantially reduce operational costs.
The widespread adoption of Artificial intelligence and Machine Learning (AI/ML) models across various fields, such as healthcare, autonomous vehicles, smart agriculture, and industrial automation, has led to a growing demand for efficient and scalable AI/ML solutions. However, as AI/ML algorithms grow more complex, their substantial memory requirements and high energy consumption pose significant challenges for deployment on resource-constrained embedded systems, such as wearable health monitors and IoT devices. To this end, various techniques, such as model pruning, knowledge distillation, quantization of model parameters, and employing approximate arithmetic operators, are commonly explored to overcome these challenges [1] .
The AI revolution, fueled by effective Deep Learning approaches, has seen a recent shift towards processing the AI workloads closer to the user, at the Edge. This paper addresses the instrumental role of system-level design space exploration (DSE) methods for achieving efficient inference of deep-learning models on resource-constrained devices at the Edge.
Artificial Neural Networks (ANNs) appear to be one of the technological revolutions of recent human history. The capability of such systems does not come at a low cost, which led researchers to develop more and more efficient techniques to implement them. Optimization approaches have been developed, such as pruning and quantization, leading to reduced memory and computation requirements. Furthermore, such approaches are adapted to the specific hardware platform features to further increase efficiency. To improve it further, the HW programmability can be traded off in favor of more specialized custom HW ANN accelerators. In this education abstract, we illustrate how optimizing operations execution at different levels, from SW to HW, can improve the efficiency of ANN execution.
Convolutional Neural Networks (CNNs) require costly hardware for real-time processing due to massive multiplications. In video-based inference, many pixels remain unchanged between frames. By skipping these operations and reusing previous results, CNNs can be accelerated without accuracy loss. However, efficient implementation of sparse operations is challenging due to the need for change detection and sparse operations in each layer. To address this, we propose Temporal RegionDrop, which performs change detection once per frame and uses a common mask to reduce conversion costs between sparse and dense formats. This method is applicable to all CNNs without retraining. We evaluate our RegionDrop on AGX Orin GPU with ChangeDetectionNet, MOT15, and JHMDB datasets. RegionDrop achieves a 2.0x-6.5x speedup with minimal accuracy loss.
This paper presents ACPO: An AI-Enabled Compiler Framework; a novel framework that provides LLVM with simple and comprehensive tools to enable employing ML models for different optimization passes. We showcase a couple of use cases of ACPO by ML-enabling the Loop Unroll (LU) and Function Inlining (FI) passes and experimental results reveal that by including both models, ACPO can provide a combined speedup of 2.4% on Cbench when compared with LLVM’s O3.
With the increased interest in Quantum Machine Learning (QML), the integration of classical data into quantum systems presents unique challenges and opportunities. The class "Primer on Data in Quantum Machine Learning" delves into the foundational concepts and advanced techniques of embedding classical data into quantum states, a critical process for enhancing the performance of quantum algorithms. By exploring various quantum embedding methods and understanding their strengths and limitations, participants will gain a comprehensive understanding of the impact quantum embeddings can have on machine learning applications. This lesson will cover the following concepts: Fundamental Concepts of Quantum Machine Learning, Limits of NISQ devices and Computing in the NISQ era, Embeddings for QML, and Practical effects of embeddings. The understanding of these topics should provide a better understanding of the importance and effect of embeddings on the overall performance of QML in the NISQ era.
Convolutional neural networks are computationally intensive. A single inference can require billions of multiply/accumulate operations. In the datacenter, where ample power, space, and cooling are available, high powered CPUs or GPUs can be used. However, in the embedded space, edge systems often do not have the power or compute capability to perform these calculations. Further, edge applications often have hard real-time requirements. This combination necessitates the use of hardware accelerators for performing inferencing calculations. A bespoke accelerator will deliver the highest performance and efficiency, outperforming configurable IP by an order of magnitude or more. But transforming an algorithm developed in a machine learning framework, often in Python, into silicon can be a daunting task. Traditional hardware design is performed at the register transfer level (RTL). When designing in RTL, hardware engineers describe every register, every operator, and every connection in the design. At this level of detail, it is impractical for the designer to explore significantly different implementation architectures. High-Level Synthesis (HLS) allows a hardware engineer to operate at a higher level of abstraction – specifically, at the algorithmic level – and use the facilities of the HLS tool to automate the conversion of the algorithm description into synthesizable RTL. This enables a broad set of architectures to be created and evaluated. And at the algorithmic level, the description of the hardware is much closer to the original Python, making the conversion faster, easier, and more natural.
Myelinated Retinal Nerve Fibers (MRNFs) can be observed on fundus photography. This has the appearance of white striated patches with poorly defined borders caused by a developmental error leading to myelination of retinal ganglion cell fibers by ectopic oligodendrocytes. The prevalence of MRNFs is low, ranging from 0.34% [1] to 0.98% [2], and there is no variation according to the origin or sex of the patients. Apart from associated pathologies, this type of fiber remains stable over time [2]. It is considered as a rare and mostly benign congenital anomaly. The objective is to show that in young children, the presence of extensive myelin fibers should be monitored more regularly by ophthalmologists.
