The NP-complete combinatorial optimization task of assigning offsets to a set of buffers with known sizes and lifetimes so as to minimize total memory usage is called dynamic storage allocation (DSA). Existing DSA implementations bypass the theoretical state-of-the-art algorithms in favor of either fast but wasteful heuristics, or memory-efficient approaches that do not scale beyond one thousand buffers. The "AI memory wall", combined with deep neural networks' static architecture, has reignited interest in DSA. We present idealloc, a low-fragmentation, high-performance DSA implementation designed for million-buffer instances. Evaluated on a novel suite of particularly hard benchmarks from several domains, idealloc ranks first against four production implementations in terms of a joint effectiveness/robustness criterion.

Vision Transformer (ViT) models which were recently introduced by the transformer architecture have shown to be very competitive and often become a popular alternative to Convolutional Neural Networks (CNNs). However, the high computational requirements of these models limit their practical applicability especially on low-power devices. Current state-of-the-art employs approximate multipliers to address the highly increased compute demands of DNN accelerators but no prior research has explored their use on ViT models. In this work we propose TransAxx, a framework based on the popular PyTorch library that enables fast inherent support for approximate arithmetic to seamlessly evaluate the impact of approximate computing on DNNs such as ViT models. Using TransAxx we analyze the sensitivity of transformer models on the ImageNet dataset to approximate multiplications and perform approximate-aware finetuning to regain accuracy. Furthermore, we propose a methodology to generate approximate accelerators for ViT models. Our approach uses a Monte Carlo Tree Search (MCTS) algorithm to efficiently search the space of possible configurations using a hardware-driven hand-crafted policy. Our evaluation demonstrates the efficacy of our methodology in achieving significant trade-offs between accuracy and power, resulting in substantial gains without compromising on performance.

The advancement towards B5G/6G relies on the synthesis of connect-compute platforms and their use in highly heterogeneous clusters featuring hardware accelerators. While these accelerators offer improved computational efficiency, sill, they make development, deployment, and orchestration of services more complex, with limited.exibility, and necessitate domain-speci.c knowledge. In AI@EDGE we are targeting seamless integration of such diverse platforms for executing AI-related tasks. This paper focuses on acceleration aspects and presents a MEC system that facilitates AI servicing over a cluster of FPGA, GPU, and CPU nodes. To this end, we develop our custom tools for generating multi-variant AI models, informative function descriptors,.exible MEC orchestrators, and runtime resource managers. The results show successful interoperability, with generic Python models getting deployed/migrated across distinct platforms for performance gains in the area of 10x.

In digital communications, accurately modeling and replicating the channel effects between a transmitter and receiver is essential to assess their impact on transmitted data. Access to a real-time channel emulator is therefore crucial in the development of telecommunication systems. This paper presents a configurable, flexible, and high-performance FPGA-based channel emulator that supports multiple channel models, including Doppler, AWGN, and multi-path Fading. We explore bit-width precision in the components and present tradeoffs between quality and resource utilization. Our exploration demonstrates similar to 25%-similar to 45% savings in various resources while maintaining the accuracy of the channel effects and operating at a high throughput of 500 Msps.

This letter proposes an online energy management framework for smart buildings. Two low-complexity estimators for thermal comfort and electrical load are investigated and integrated into an optimization framework. The proposed framework optimizes operations of a battery system and multiple heating, ventilation, and air conditioning (HVAC) systems to minimize energy consumption and power peak while maximizing occupants’ thermal comfort. The effectiveness of the proposed framework is demonstrated using data measured in actual campus buildings in terms of system costs, thermal comfort, and computational complexity. The results show that the proposed framework can reduce electricity costs by 5.7% compared to the baseline.

Developing embedded software applications is a challenging task, chiefly due to the limitations that are imposed by the hardware devices or platforms on which they operate, as well as due to the heterogeneous non-functional requirements that they need to exhibit. Modern embedded systems need to be energy efficient and dependable, whereas their maintenance costs should be minimized, in order to ensure the success and longevity of their application. Being able to build embedded software that satisfies the imposed hardware limitations, while maintaining high quality with respect to critical non-functional requirements is a difficult task that requires proper assistance. To this end, in the present paper, we present the SDK4ED Platform, which facilitates the development of embedded software that exhibits high quality with respect to important quality attributes, with a main focus on energy consumption, dependability, and maintainability. This is achieved through the provision of state-of-the-art and novel quality attribute-specific monitoring and optimization mechanisms, as well as through a novel fuzzy multi-criteria decision-making mechanism for facilitating the selection of code refactorings, which is based on trade-off analysis among the three main attributes of choice. Novel forecasting techniques are also proposed to further support decision making during the development of embedded software. The usefulness, practicality, and industrial relevance of the SDK4ED platform were evaluated in a real-world setting, through three use cases on actual commercial embedded software applications stemming from the airborne, automotive, and healthcare domains, as well as through an industrial study. To the best of our knowledge, this is the first quality analysis platform that focuses on multiple quality criteria, which also takes into account their trade-offs to facilitate code refactoring selection.

In response to the growing complexity of Deep Neural Network (DNN) models, the paradigm of approximate computing has emerged as a compelling approach to strike a balance between computational efficiency and model accuracy. Approximate computing involves intentionally allowing errors using inexact computations, exploiting the inherent resilience of many applications to such inaccuracies. While current state-of-the-art approaches leverage approximate multipliers for approximate DNN accelerators, there has been a notable difficulty in emulating such DNNs on common frameworks such as PyTorch, as they do not inherently support inexact arithmetic. This paper introduces a seamless PyTorch plugin, to evaluate inexact DNN models while having GPU acceleration to enable rapid emulation. Additionally, we support approximate-aware retraining and perform evaluation on several popular convolutional neural networks on Cifar10 and ImageNet datasets.

Modeling and replicating the channel effects between a transmitter and receiver, is crucial in any telecommunication system, in order to evaluate the impact on the transmitted data. Therefore the necessity of having access to a real time channel emulator is vital. In order to meet these constraints, in this work we present a flexible, high performance and resource efficient FPGA-based channel emulator, focusing on fading models. Additionally, we perform a trade-off analysis between bit accuracy, resources utilization and system’s accuracy in terms of Mean Relative Error and the Probability Density Function. Our study shows that by using a circuit with reduced bit-widths, we can achieve 24%–46% reduction in various resources, while preserving the algorithmic accuracy of the channel emulator and operating at a throughput of 500Msps.

In recent decades, increasing ultrasound frame rates has been the main motivation behind many novel ultrasound imaging applications [1]–[3]. With this work, we propose an efficient ultrafast FPGA beamformer that applies coherent compounding, through a delay-reuse optimization.

Over the last years the rapid growth Machine Learning (ML) inference applications deployed on the Edge is rapidly increasing. Recent Internet of Things (IoT) devices and microcontrollers (MCUs), become more and more mainstream in everyday activities. In this work we focus on the family of STM32 MCUs. We propose a novel methodology for CNN deployment on the STM32 family, focusing on power optimization through effective clocking exploration and configuration and decoupled access-execute convolution kernel execution. Our approach is enhanced with optimization of the power consumption through Dynamic Voltage and Frequency Scaling (DVFS) under various latency constraints, composing an NP-complete optimization problem. We compare our approach against the state-of-the-art TinyEngine inference engine, as well as TinyEngine coupled with power-saving modes of the STM32 MCUs, indicating that we can achieve up to 25.2% less energy consumption for varying QoS levels.

Today, there is an ever-increasing number of workloads pushed and executed on the Cloud. Data center operators and Cloud providers have embraced application co-location and multi-tenancy as first-class system design concerns to effectively serve and manage these huge computational demands. In addition, the continuous advancements in the computers’ hardware technology have made it possible to seamlessly leverage heterogeneous pools of physical machines in data center environments. Even though current modern Cloud schedulers and orchestrators adopt application-aware policies to achieve automation of time-consuming management tasks at scale, e.g., resource provisioning, they still rely on coarse-grained system metrics, such as CPU and/or memory utilization to place incoming applications, thus, not considering (1) interference effects that are provoked by co-located tasks, and (2) the impact on performance caused by the diversity of heterogeneous systems’ characteristics. The lack of such knowledge in existing state-of-the-art orchestration solutions results in their inability to perform efficient allocations, which negatively impacts the overall latency distribution delivered by the infrastructure. In this paper, to alleviate this inefficiency, we present a machine learning (ML) based Cloud orchestration extension that takes into account both resource interference and heterogeneity. The framework adequately schedules data-analytics applications on a pool of heterogeneous resources. We evaluate our proposed solution on different application mixes and co-location scenarios. We show that the proposed framework improves the tail latency of the distribution of the deployed applications by up to 3.6x compared to the state-of-the-art Kubernetes scheduler.

High-Level Synthesis (HLS) has democratized FPGAs by enabling high-level device programmability and rapid microarchitecture customization through the use of directives. Nevertheless, the manual selection of the appropriate directives, i.e. the annotations included in the high-level source code to instruct synthesis process, is a difficult task for programmers without a hardware background. In this paper, we present CollectiveHLS, an ultra-fast knowledge-based HLS design optimization method that automatically extracts the most promising directive configurations and applies them to the original source code. The proposed optimization scheme is a fully data-driven approach for generalized HLS tuning, as it is not based on QoR models or meta-heuristics. We design, implement, and evaluate our method with more than 100 applications of Machsuite, Rodinia, and GitHub on an ZCU104 FPGA. We achieve an average geometric mean speedup of x14.1 and x10.5 compared to the unoptimized, i.e., without HLS directives and optimized designs, a high design feasibility score, and an average inference latency of 38ms.

The increasing popularity of LLM-based chatbots combined with their reliance on power-hungry GPU infrastructure forms a critical challenge for providers: minimizing energy consumption under Service-Level Objectives (SLOs) that ensure optimal user experience. Traditional energy optimization methods fall short for LLM inference due to their autoregressive architecture, which renders them incapable of meeting a predefined SLO without energy overprovisioning. This autoregressive nature however, allows for iteration-level adjustments, enabling continuous fine-tuning of the system throughout the inference process. In this letter, we propose a solution based on iteration-level GPU Dynamic Voltage Frequency Scaling (DVFS), aiming to reduce the energy impact of LLM serving, an approach that has the potential for more than 22.8% and up to 45.5% energy gains when tested in real world situations under varying SLO constraints. Our approach works on top of existing LLM hosting services, requires minimal profiling and no intervention to the inference serving system.

As Large Language Models (LLMs) gain traction, their reliance on power-hungry GPUs places ever-increasing energy demands, raising environmental and monetary concerns. Inference dominates LLM workloads, presenting a critical challenge for providers: minimizing energy costs under Service-Level Objectives (SLOs) that ensure optimal user experience. In this paper, we present throttLL'eM, a framework that reduces energy consumption while meeting SLOs through the use of instance and GPU frequency scaling. throttLL'eM features mechanisms that project future KV cache usage and batch size. Leveraging a Machine-Learning (ML) model that receives these projections as inputs, throttLL'eM manages performance at the iteration level to satisfy SLOs with reduced frequencies and instance sizes. We show that the proposed ML model achieves R^2 scores greater than 0.97 and miss-predicts performance by less than 1 iteration per second on average. Experimental results on LLM inference traces show that throttLL'eM achieves up to 43.8% lower energy consumption and an energy efficiency improvement of at least 1.71× under SLOs, when compared to NVIDIA's Triton server.

Today, video analytics are becoming extremely popular due to the increasing need for extracting valuable information from videos available in public sharing services through camera-driven streams in IoT environments. To avoid data communication overheads, a common practice is to have computation close to the data source rather than Cloud offloading. Typically, video analytics are organized as separate tasks, each with different resource requirements (e.g., computational- vs. memory-intensive tasks). The serverless computing paradigm forms a promising approach for mapping such types of applications, enabling fine-grained deployment and management in a per-function, and per-device manner. However, there is a tradeoff between QoS adherence and resource efficiency. Performance variability due to function co-location and prevalent resource heterogeneity make maintaining QoS challenging. At the same time, resource efficiency is essential to avoid waste, such as unnecessary power consumption and CPU reservation. In this paper, we present Darly, a QoS-, interference- and heterogeneity-aware Deep Reinforcement Learning-based Scheduler for serverless video analytics deployments on top of distributed Edge nodes. The proposed framework incorporates a DRL agent that exploits performance counters to identify the levels of interference and the degree of heterogeneity in the underlying Edge infrastructure. It combines this information along with user-defined QoS requirements to improve resource allocations by deciding the placement, migration, or horizontal scaling of serverless functions. We evaluate Darly on a typical Edge cluster with a real-world workflow composed of commonly used serverless video analytics functions and show that our approach achieves efficient scheduling of the deployed functions by satisfying multiple QoS requirements for up to 91.6% (Profile-based) of the total requests under dynamic conditions.

Serverless workflows have emerged in Function-as-a-Service (FaaS) platforms to represent the operational structure of traditional applications. With latency propagation effects becoming increasingly prominent, step-wise resource tuning is required to address Service-Level-Objectives (SLOs). Modern processors' allowance for fine-grained Dynamic Voltage and Frequency Scaling (DVFS), coupled with serverless workflows' intermittent nature, presents a unique opportunity to reduce power while meeting SLOs. We introduce Ωkypous, an SLO-driven DVFS framework for serverless workflows. Ωkypous employs a grey-box model that predicts functions' execution latency and power under different Core and Uncore frequency combinations. Based on these predictions and the timing slacks between workflow functions, Ωkypous uses a closed-loop control mechanism to dynamically adjust Core and Uncore frequencies, thus minimizing power consumption without compromising predefined end-to-end latency constraints. Our evaluation on real-world traces from Azure, against state-of-the-art power management frameworks, demonstrates an average power consumption reduction of 16%, while consistently maintaining low SLO violation rates (1.8%), when operating under power caps.

Star trackers are crucial for satellite orientation. Improving their efficiency via reconfigurable COTS HW accommodates NewSpace missions. The current work considers SoC FPGAs to leverage both increased reprogramming and high-performance capabilities. Based on a custom sensor+FPGA system, we develop and optimize the algorithmic chain of star tracking by focusing on the acceleration of the image processing parts. We combine multiple circuit design techniques, such as low-level pipelining, word-length optimization, HW/SW co-processing, and parametric HLS+HDL coding, to fine-tune our implementation on Zynq-7020 FPGA when using real and synthetic input data. Overall, with 4-MPixel images, we achieve more than 24 FPS throughput by accelerating >95 × , at system level, while preserving the original SW accuracy and meeting the real-time requirements of the application.

Combined cyber and physical attacks on Critical Infrastructures have disastrous consequences on economies and in social well-being. Protection and resilience of CIs under combined attacks is challenging due to their complexity, reliance on ICT systems and the interdependences between different types of CIs. The PRAETORIAN framework was designed to address these challenges, by integrating components responsible for detecting both cyber and physical threats. Additionally, it forecasts how the combined attacks will evolve and their cascading effects on interdependent CIs. The PRAETORIAN framework was demonstrated based on a realistic scenario in the Zagreb airport, combining both physical and cyber attacks.

This paper presents the end-to-end integration of a software defined network (SDN) reconfigurable real-time analog-intermediate frequency-over-fiber (A-IFoF)/millimeter wave (mmWave) layout into a real mobile network operator (MNO) infrastructure. The key part of the presented infrastructure is the fully programmable gateway array (FPGA)-based A-IFoF transceiver (TxRx), equipped with active SDN functionalities based on constant monitoring of the transport link. This layout was successfully evaluated in the physical data management and application layer. Regarding the physical layer performance evaluation, error vector magnitude values of 7.3% were achieved for quadrature phase shift keying waveforms, while the uninterrupted operation of real-time services such as augmented reality/virtual reality applications was enabled by the reactive SDN-based capacity reconfiguration capabilities.

The challenging deployment of Artificial Intelligence (AI) and Computer Vision (CV) algorithms at the edge pushes the community of embedded computing to examine heterogeneous System-on-Chips (SoCs). Such novel computing platforms provide increased diversity in interfaces, processors and storage, however, the efficient partitioning and mapping of AI/CV workloads still remains an open issue. In this context, the current paper develops a hybrid AI/CV system on Intel’s Movidius Myriad X, which is an heterogeneous Vision Processing Unit (VPU), for initializing and tracking the satellite’s pose in space missions. The space industry is among the communities examining alternative computing platforms to comply with the tight constraints of on-board data processing, while it is also striving to adopt functionalities from the AI domain. At algorithmic level, we rely on the ResNet-50-based UrsoNet network along with a custom classical CV pipeline. For efficient acceleration, we exploit the SoC’s neural compute engine and 16 vector processors by combining multiple parallelization and low-level optimization techniques. The proposed single-chip, robust-estimation, and real-time solution delivers a throughput of up to 5 FPS for 1-MegaPixel RGB images within a limited power envelope of 2 W.