Hardware prefetching in processors, as an effective solution to hide the cache miss latencies, needs to be redesigned according to emerging applications behavior. Conventional prefetchers try to predict predefined patterns but are unable to prefetch a mixture of even simple detectable patterns, e.g., access pattern of convolution operation in neural networks, or randomness in workload behavior. In this paper, we expose and define new memory access patterns and propose a prefetcher capable of detecting them in addition to its capability to detect the patterns detectable by existing prefetchers in L2 caches. The proposed scheme, named Learning-based Adaptive Multi-objective Prefetcher (LAMP), not only supports intra-page patterns but also detects inter-page patterns as its unique feature. It is also able to extrapolate a pattern from one page to the other pages by detecting a relation between them, offering cross-page prefetching. The other feature is to detect random accesses to a limited memory area as well as irregular traversal of a memory address path. LAMP learns the patterns in less than or equal to three block accesses offering the minimum learning time not to miss prefetching opportunities. It adaptively adjusts the prefetching parameters and decisions to maximize efficiency via a tiny tunable learning-based engine. The tradeoff between accuracy and coverage can be tuned with the LAMP mechanism, which can (automatically) adjust the prefetch aggressiveness and depth. The evaluation demonstrates that the proposed approach surpasses existing state-of-the-art methods by achieving superior coverage and accuracy. Relative to the baseline, it improves performance by 17.3%, 27.0%, and 7.6% on the SPEC CPU2017, GAP, and CloudSuite benchmark suites, respectively. In addition, the scheme reduces cache energy consumption by lowering the proportion of ineffective prefetching requests from over 43% in prior methods to below 26%, while incurring substantially less hardware overhead.

Autonomous Driving is emerging as a paradigm shift in the way we conceive people and goods transportation. It promises to improve road safety, reduce traffic congestion, and increase overall transportation efficiency. It is made possible by a plethora of modern technologies, such as AI, low-power hardware, and complex computing. Common stages of Autonomous Driving systems are the identification of objects in the scene (Object Detection), and the ability to predict the evolution of the tracked objects' states - usually, positions and velocities (Multi-Object Tracking). In this paper, we address the issues behind high-performance Multi-Object Tracking (MOT) algorithms for a real-world urban transportation scenario. The main objective is to exploit the high-performance capabilities of NVIDIA heterogeneous embedded platforms, which are not automatically compatible with the peculiar features of these algorithms. We propose and discuss some non-trivial software design choices and implementation strategies that are needed to match the specific computational and memory access needs of MOT strategies, and the architectural opportunities of NVIDIA embedded GPGPU systems. Our code is made available as open-source. We compare our solution against a highly optimized multi-core version and show that we are able to trigger significant performance speedups (up to 7.19x) in all power-modes, sometimes even surpassing the CPU reference with the GPU in a lower-power operating mode.

Peristaltic pumps (PP), widely acknowledged for their benefits in pharmaceutical contexts, face challenges in achieving optimal dosing accuracy. This investigation contributes novel insights for the improvement of dosing precision, identifying how to apply AI models, specifically focusing on Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) neural networks over a realistic span of target volumes. To provide a more accurate representation of real-world performance, we consider a modified root mean square error metric (RMSEPP) that directly compares dispensed volumes to target volumes. Based on this the study delves into two main methodologies: an iterative retraining method, called Online Training, and Pre-trained approach. Online Training shows best results, especially for volumes below 1.0 ml, achieving 38.4% improvement in RMSEPP and 31.6% in standard deviation (STD). Pre-trained models are faster and exhibit promising outcomes especially for volumes above 1.0 ml, with a three-features approach delivering the best performance (13.8% and 4.6% improvements in RMSEPP and STD, respectively). Overall, the findings highlight the effectiveness of iterative learning techniques, particularly for smaller dosage amounts, which complements the good performance of non-AI approaches for larger ones.

This study investigates how ensemble learning techniques can be employed for enhancing peristaltic pump accuracy in pharmaceutical manufacturing, and demonstrates significant accuracy improvements through the novel E-AR implementation, with gains of up to 53.93% at 0.3 ml volume compared to 47% achievable with single models. To establish the foundation for ensemble methods evaluation, we first conduct a comprehensive validation of traditional Adaptive Dosing Control System (ADCS) across an extended volume range (0.1-2.0 ml), demonstrating base performance improvements. In this investigation, we develop a novel offline performance indicator enabling rapid assessment of compensation strategies without extensive physical testing, showing strong correlation with actual measurements. These premises enable a thorough investigation of various ensemble configurations, revealing volume-dependent performance patterns where different models excel under specific conditions, suggesting that practical applications may benefit from volume-specific model selection. The comparison with a very accurate reference mechanical pump, demonstrates that our ADCS solutions achieve comparable or superior performance across most volumes while maintaining the cost-effectiveness. Statistical validation via a multi-dimensional framework confirms the significance of these improvements through multiple complementary tests: paired t-tests showing significant mean differences with p $\leq 0.001$ , Mann-Whitney U tests confirming distributional shifts, Levene tests demonstrating variance modifications with statistics up to 801.65, and mixed linear model analysis with F-statistics ranging from 0.004 to 1497.75 confirming global effects.

We publish a comprehensive dataset of peristaltic pump dosing outputs in pharmaceutical manufacturing, where accuracy is crucial for drug quality and patient safety, consisting of 149,847 measurements spanning volumes from 0.1 to 2.0 ml. An industrial filling system was used to acquire data under controlled conditions, using calibrated weighing equipment. To the best of our knowledge, this is the first dataset documenting pump behavior across such a wide range of volumes. The dataset aims to constitute a solid tool that enables investigation from short-term precision to long-term stability standpoints, providing detailed insights into peristaltic pump behavior under various operating conditions. Additionally, the dataset incorporates compensation outcomes across multiple volumes, documenting both statistical and AI-based compensation strategies, thus exemplifying how the statistical behavior of the dosing can change in response to some compensation strategies aimed to improve dosing accuracy. This resource directly addresses pharmaceutical industry needs by supporting optimization of quality control systems and validation of novel compensation strategies.

The constant growth of workload size in modern applications is making address translation a performance bottleneck. In principle, increasing the virtual page size could be advantageous, as it would allow each cached address translation to cover a larger memory space. Nevertheless, the utilization of larger pages introduces challenges, such as issues related to memory fragmentation and physical page management. In this paper, we present Decoupled Virtual Address Spaces (DeVAS), a virtual memory proposal that enables the decoupling of address translation and memory allocation, by allowing different sizes for virtual and physical pages, aiming to exploit the benefits of both. DeVAS introduces an intermediate virtual address space allocated by the Operating System employing larger pages (e.g., 2MiB), corresponding to the virtual page size seen by the processor, and a memory controller extension devoted to their allocation in physical memory at a smaller granularity (e.g., 4KiB). We show that DeVAS achieves an average 1.13x performance improvement over a traditional configuration (i.e., with 4KiB pages) with no specific architectural modifications and for memory-intensive benchmarks. Moreover, DeVAS strategy enables architectural modifications for increasing performance, such as simplified/optimized TLB structure and L1-cache design flexibility. When considering these adjustments, DeVAS achieves a speedup of up to 1.20x compared to the same reference. Furthermore, it matches and even edges the performance of an ideal (i.e., not implementable in practice) virtual memory configuration based on 2MiB pages only.

Object tracking is an important and central aspect of autonomous driving, as it underlies the obstacle detection and avoidance systems of any type of autonomous vehicles. A widely used method for tracking is based on Kalman filters, both for linear and non-linear cases, with different computational burden. Unfortunately, object tracking algorithms are computationally intensive, and they may not easily meet the efficiency and responsiveness requirements of real-time applications such as autonomous driving. This issue motivates ad-hoc investigations to speed up the computation and make Kalman filtering available even within limited computational power. This paper carry out a performance evaluation of a Kalman filter based object tracking system taken from a real tramway use-case, and aims at improving its performance efficiency by leveraging parallelization. In particular, this work analyzes the possibilities of execution parallelization on multi-core processors, proposing a target-specific optimization approach and comparing the obtained results, then summing them in general lessons learned. Our technique achieves up to 80% reduction of single frame processing time in the most crowded cases.

gem5 is a popular architectural simulator, for both academic and industrial researchers. It can be used in two configurations: Full System mode and Syscall Emulation mode. The former requires running a real kernel to achieve realistic results, at the cost of increased user effort. In contrast, the latter emulates operating system functionalities, which improves usability but is more prone to producing less accurate results. Due to the absence of a genuine kernel in Syscall Emulation mode, the simulator model of virtual address translation remains inaccurate. In the current gem5 version (v23.0.1.0), the address translation is performed through the lookup of a flat structure that stores all the virtual-to-physical mappings. However, this approach does not reflect the behaviour of a real multi-level page table, lacking the additional latency associated with page walks. In this paper, we present our implementation of the page walk functionality in Syscall Emulation mode for the RISC-V ISA. We show how our page walker affects the performance of simulated benchmarks and also its sensitivity on the TLB size. Furthermore, we make our work publicly available, inviting fellow researchers to utilize and build upon the model to suit their specific requirements.

During the last few years, hardware accelerators have been gaining popularity thanks to their ability to achieve higher performance and efficiency than classic general-purpose solutions. They are fundamentally shaping the current generations of Systems-on-Chip (SoCs), which are becoming increasingly heterogeneous. However, despite their widespread use, a standard, general solution to manage them while providing speed and consistency has not yet been found. Common methodologies rely on OS mediation and a mix of user-space and kernel-space drivers, which can be inefficient, especially for fine-grained tasks. This paper addresses these sources of inefficiencies by proposing an ISA eXtension for Integrated Accelerator Management (IXIAM), a cost-effective HW-SW framework to control a wide variety of accelerators in a standard way, and directly from the cores. The proposed instructions include reservation, work offloading, data transfer, and synchronization. They can be wrapped in a high-level software API or even integrated into a compiler. IXIAM features also a user-space interrupt mechanism to signal events directly to the user process. We implement it as a RISC-V extension in the gem5 simulator and demonstrate detailed support for complex accelerators, as well as the ability to specify sequences of memory transfers and computations directly from the ISA and with significantly lower overhead than driver-based schemes. IXIAM provides a performance advantage that is more evident for small and medium workloads, reaching around $90\times $ in the best case. This way, we enlarge the set of workloads that would benefit from hardware acceleration.

Autonomous vehicles are nowadays gaining popularity in many different sectors, from automotive to aviation, and find application in increasingly complex and strategic contexts. In this domain, Obstacle Detection and Avoidance Systems (ODAS) are crucial and, since they are safety-critical systems, they must employ fault-detection and management techniques to maintain correct behavior. One of the most popular techniques to obtain a reliable system is the use of redundancy, both at the hardware and at the software levels. With the objective of improving fault-detection while producing little impact on the programmability of the system, this paper introduces a general and lightweight monitoring technique based on a user-directed observer design pattern, which aims at monitoring the validity of predicates over state variables of the algorithms in execution. This can increase the fault-detection capability and even anticipate the detection time of some faults that would be caught by replication only at later times. Results are evaluated on a real-world use-case from the railway domain, and show how the proposed fault-detection mechanism can increase the overall reliability of the system by up to 24.4% compared to replication alone in case of crowded scenarios over the entire tracking process, and up to 43.9% in specific phases.

Peristaltic pumps play a crucial role in the pharmaceutical industry, offering advantages such as reduced cross-contamination risks and ease of use. However, their dosing precision often lags behind other devices like volumetric pumps. This study investigates the underlying causes of this phenomenon and proposes effective mitigation strategies to enhance accuracy. Notably, two novel aspects are explored: the underlying causes of dosing variation and compensation systems on precision filling. Through comprehensive analysis, the impact of product temperature on accuracy is unveiled, resulting mainly from variations that alter the elastic properties of the pipe material and lead to significant deviations in dosed volume. Therefore, temperature stabilization becomes imperative for optimal performance. Additionally, the Adaptive Dosing Control System (ADCS) based on time series prediction is introduced, enabling real-time compensation of volume delivery. The filling system is considered as a black box, allowing potential application of these findings on other similar industrial setups. Extensive experiments on state-of-the-art robotic production lines validate the ADCS’s stability and effectiveness, demonstrating up to a 30% improvement in accuracy. In conclusion, this research sheds light on the critical relationship between product temperature and peristaltic pump dosing, while the ADCS represents an advancement in precision filling technology. These results hold potential for enhancing precision, reducing wastage, and improving product quality in the pharmaceutical industry and other precision filling applications.

The growing demand for deep learning applications has led to the design and development of several hardware accelerators to increase performance and energy efficiency. In particular, convolutional accelerators are among those receiving the most attention due to their applicability in many fields. Another aspect that is gaining increasing attention is the use of a shared virtual address space between processor and accelerators. It can provide several advantages such as programmability and security. The use of a shared address space relies on a time-consuming IOMMU to satisfy address translation requests. In this work, we analyze convolutional workloads in convolutional accelerators, identifying the sensitivity of performance to IOMMU activity. Additionally, based on the analysis done on convolutional workloads, we propose the use of dedicated accelerator registers (Translation Registers) to reduce costly IOMMU accesses. Translation Registers allow reducing execution time by about 20% and the energy consumption related to address translation up to about 55%.

Nowadays, convolutional neural networks are among the most widely used types of deep learning networks thanks to their usefulness in many application domains. There are many efforts to find methods to increase their training and inference performance and efficiency. One of the most widely used technique to implement convolution consists of flattening tensors into 2D matrices and carrying out the operation through a matrix-matrix multiplication routine, which has highly optimized implementations in high-performance libraries. However, this kind of approach uses extra time and memory to transform and store the tensors involved. For this reason, direct convolution is becoming increasingly popular. Direct convolution can be implemented as a series of nested loops iterating over tensor dimensions and it does not require extra memory. In this work, we evaluate on various multi-core CPUs the performance and scalability effects deriving from different parallelization strategies, loop organizations, and SIMD-vectorization approaches with different compilers in relation with architectural aspects. We discuss each parameter thoroughly and distill our findings in a set of heuristics that can be used to quickly achieve a high-performance implementation in accordance to the underlying hardware and the characteristics of the convolutional layer at hand. By adopting a per-layer approach, we increase performance up to 60-70% compared to a static implementation for all the layers. Moreover, our results are comparable, or even better (up to $1.67\times $ speedup) than matrix-matrix multiplication-based convolution in a multi-core system.

This work covers the PHAST Library's employment, a hardware-agnostic programming library, to a real-world application like the Caffe framework. The original implementation of Caffe consists of two different versions of the source code: one to run on CPU platforms and another one to run on the GPU side. With PHAST, we aim to develop a single-source code implementation capable of running efficiently on CPU and GPU. In this paper, we start by carrying out a basic Caffe implementation performance analysis using PHAST. Then, we detail possible performance upgrades. We find that the overall performance is dominated by few 'heavy' layers. In refining the inefficient parts of this version, we find two different approaches: improvements to the Caffe source code and improvements to the PHAST Library itself, which ultimately translates into improved performance in the PHAST version of Caffe. We demonstrate that our PHAST implementation achieves performance portability on CPUs and GPUs. With a single source, the PHAST version of Caffe provides the same or even better performance than the original version of Caffe built from two different codebases. For the MNIST database, the PHAST implementation takes an equivalent amount of time as native code in CPU and GPU. Furthermore, PHAST achieves a speedup of 51% and a 49% with the CIFAR-10 database against native code in CPU and GPU, respectively. These results provide a new horizon for software development in the upcoming heterogeneous computing era.

Different technologies and approaches exist to work around the performance portability problem. Companies and academia work together to find a way to preserve performance across heterogeneous hardware using a unified language, one language to rule them all. Intel's oneAPI appears with this idea in mind. In this article, we try the new Intel solution to approach heterogeneous programming, choosing machine learning as our case study. More precisely, we choose Caffe, a machine learning framework that was created six years ago. Nevertheless, how would it be to make Caffe again from the beginning, using a fresh new technology like oneAPI? In terms of not only the ease of programming‐because only one source code would be needed to deploy Caffe to CPUs, GPUs, FPGAs, and accelerators (platforms that oneAPI currently supports)‐but also performance, where oneAPI may be capable of taking advantage of specific hardware automatically. Is Intel's oneAPI ready to take the leap?

In recent years, the limits of the multicore approach emerged in the so-called “dark silicon” issue and diminishing returns of an ever-increasing core count. Hardware manufacturers, out of necessity, switched their focus to accelerators, a new paradigm that pursues specialization and heterogeneity over generality and homogeneity. They are special-purpose hardware structures separated from the CPU with aspects that exhibit a high degree of variability. We define a taxonomy based on fourteen of these aspects, grouped in four macro-categories: general aspects, host coupling, architecture, and software aspects. According to it, we categorize around 100 accelerators of the last decade from both industry and academia, and critically analyze emerging trends. We complete our discussion with throughput and efficiency figures. Then, we discuss some prominent open challenges that accelerators are facing, analyzing state-of-the-art solutions, and suggesting prospective research directions for the future.

Thanks to the promised improvements in performance and energy efficiency, hardware accelerators are taking momentum in many computing contexts, both in terms of variety and relative weight in the silicon area of many chips. Commonly, the way an application interacts with these hardware modules has many accelerator-specific traits and requires ad-hoc drivers that usually rely on potentially expensive system calls to manage accelerator resources and access orchestration. As a consequence, driver-based interfacing is far from uniform and can expose high latency, limiting the set of tasks suitable for acceleration. In this paper, we propose a uniform and low-latency interface based on Instruction Set Architecture (ISA) extension. All the previous studies that proposed extensions, were deeply tailored to address a single accelerator. One of the biggest disadvantages of those methods is their inability to scale. Adding more of these accelerators to one System-on-Chip (SoC) would result in ISA bloat, increasing power consumption and complexifying the decoding phase proportionally. Our proposed framework consists of a six-instruction ISA extension and the corresponding architectural support that implements the interface abstraction and the reservation logic at the hardware level. Our proposal allows controlling a broad class of integrated accelerators directly from the CPU. The proposed framework is ISA-independent, which means that it is applicable to all the existing ISAs. We implement it on the gem5 simulator by extending the RISC-V ISA. We evaluate it by simulating three compute-intensive accelerators and comparing our interfacing with a conventional driver-based one. The benchmarks highlight the performance benefits brought by our framework, with up to 10.38x speed up, as well as the ability to seamlessly support different accelerators with the same interface. The speed up advantage of our technique diminishes as the granularity of the workloads increases and the overhead for driver-based accelerators becomes less important. We also show that the impact of its hardware components on chip area and power consumption is limited.

SummaryHeterogeneous architectures proved successful in achieving unprecedented performance and energy‐efficiency. However, taking advantage of these diverse processing elements is still hard. Programmers need to code through the different approaches suitable for each target architecture and need to decide the distribution of activities on the different resources. The majority of current frameworks focuses on either performance or productivity. The former mainly provides low‐level target‐specific programming interfaces, and the latter offers high‐level tools that often fail in achieving high‐performance. In both cases, the design is usually data‐parallel, as task‐parallelism is not supported. In this work, we propose a task‐based solution within the data‐parallel heterogeneous single‐source PHAST library. Tasks can be coded in a target‐agnostic fashion, can be compiled and parallelized on multi‐core CPUs and NVIDIA GPUs automatically and support the choice of the execution platform at runtime. We evaluate the capabilities of the proposed task‐directed acyclic graph support in case of an extensive set of randomly generated task‐based applications with different sizes and characteristics. We compare it against a SYCL implementation in terms of performance and complexity metrics, highlighting that PHAST achieves about 1.56× and 2.60× speedup over SYCL for multi‐core CPU and GPU, respectively, while improving also code complexity metrics.