In the search for performance, in-order execution cannot expect to prevail as older long latency instructions prevent younger ones from issuing. Although stall-on-use processors allow independent instructions to issue in the shadow of a cache miss, the compiler cannot always find enough independent work to keep pipeline resources busy. In this paper, we study how both value prediction based on address prediction and direct value prediction can be built into an in-order pipeline to unlock significant performance. We further show that the in-order execution property provides advantages in that the pipeline may speculate aggressively without suffering from any recovery penalty. Finally, we combine this data speculation infrastructure with a reworked cache hierarchy that relies on a fast first level cache that can be written speculatively. We show that such an in-order pipeline can reach a performance level that is comparable to an equally - although moderately - wide out-of-order processor, without requiring support for partial out-of-order execution such as out-of-order memory hazard handling or full-fledged register renaming. Overall, we increase the performance of a 32-entry scoreboard, 4-issue in-order processor based on a scaled up Open Hardware Group CVA6 by 38.4% (geomean), achieving 86.7% and 46.3% of the gains brought by comparable out-of-order processors featuring 32/16-entry and 64/32-entry Reorder Buffer and scheduler, respectively.

This work presents a floating-point Fused Dot Product Add operator designed for two use cases: mixed-precision matrix multiply add for deep learning and single-precision arithmetic for numerical computing. We build on classic doubleword techniques to emulate FP32 arithmetic as the unevaluated sum of lower-precision floating-point numbers. Specifically, we introduce E9S12, a 9-bit exponent, 12-bit significant, nonnormalized floating-point format, to represent either the FP16 and BF16 multiplicands or the results of the FP32 multiplicand decomposition inside a Fused Dot Product Add operator. The resulting operator correctly rounds the FP16 products accumulated in FP32 and also matches the standard FP32 FMA arithmetic when the size-4 E9S12 dot product operates on decomposed FP32 multiplicands. We evaluated the implementation of this combined FP16, BF16 and FP32 Dot Product Add operator and compare it to correctly rounded Fused Dot Product Add operators by synthesis for a 4nm technology node.

We revisit 1D Fast Fourier Transforms (FFT) implementation approaches in the context of compute units composed of multiple cores with SIMD ISA extensions and sharing a multi-banked local memory. A main constraint is to spare use of local memory, which motivates us to use in-place FFT implementations and to generate the twiddle factors with trigonometric recurrences. A key objective is to maximize bandwidth of the multi-banked local memory system by ensuring that cores issue maximum-width aligned non-temporal SIMD accesses. We propose combining the SIMD lane-slicing and sample partitioning techniques to derive multicore FFT implementations that do not require matrix transpositions and only involve one stage of bit-reverse unscrambling. This approach is demonstrated on the Kalray MPPA3 processor compute unit, where it outperforms the classic six-step algorithm for multicore FFT implementation.

Ce chapitre présente la conception du processeur pluri-cœur MPPA de troisième génération Kalray, dont les objectifs sont de combiner l’évolutivité des performances des GPGPU, l’efficacité énergétique des cœurs DSP et les capacités d’E/S des dispositifs FPGA. Ce processeur implémente une architecture à pluri-cœurs ciblant des systèmes cyberphysiques augmentés de capacités d’apprentissage automatique et une prise en charge moderne de la cybersécurité.

This article explores architectures of exact (correctly rounded) fused dot product and add operators suitable for the FP32 and FP64 binary floating-point representations with subnormal support, and other representations with a wide dynamic range such as bfloat16. The exact summation of terms before rounding requires a full-size accumulator, and this work discusses techniques to compress the identical bits of this accumulator. This requires the computation of the relative shift amounts of the terms, which is formulated as a parallel prefix algorithm, allowing for a low-latency implementation. Architectural options for the exact fused dot product and add operators with up to 16 products for FP32, FP64 and mixed-precision BF16 to FP32 are evaluated using the TSMC 16FFC technology node.

Low bit-width floating-point formats appear as the main alternative to 8-bit integers for quantized deep learning applications. We propose an architecture for exact dot product accumulate operators and compare its implementation costs for different 8-bit floating-point formats: FP8 with five exponent bits and two fraction bits (E5M2), FP8 with four exponent bits and three fraction bits (E4M3), and Posit8 formats with different exponent sizes. The front-ends of these exact dot product accumulate operators take 8-bit multiplicands, expand their full-precision products to fixed-point, and sum terms into wide accumulators. The back-ends of these operators round down the wide accumulators contents first to FP32 and then to one of the 8-bit floating-point formats. We synthesize the proposed 8-bit floating-point exact dot product accumulate operators targeting the TSMC 16FFC node and compare their area and power to a baseline of operators with FP16 and INT8 multiplicands.

We propose a hardware operator to decompress Posit8 representations with exponent sizes 0, 1, 2, 3 to the IEEE 754 binary 16 (FP16) representation. The motivation is to leverage the tensor units of a manycore processor that already supports FP16.32 matrix multiply-accumulate operations for deep learning inference. According to our experiments, adding instructions to decompress Posit8 into FP16 numbers would enable to further reduce the footprint of deep neural network parameters with an acceptable loss of accuracy or precision. We present the design of our decompression operator and compare it to lookup-table implementations for the technology node of the targeted processor.

Cache and cache-coherence are major aspects of today's high performance computing. A cache stores data as cachelines of fixed size, and coherence between caches is guaranteed by the cache-coherence protocol which operates on fixed size coherency-blocks. In such systems cache-lines and coherencyblocks are usually the same size and are relatively small, typically 64 bytes. This size choice is a trade-off selected for generalpurpose computing: it minimizes false-sharing while keeping cache-maintenance traffic low. False-sharing is considered an unnecessary cache-coherence traffic and it decreases performances. However, for dedicated accelerator this trade-off may not be appropriate: hardware in charge of cache-coherence is expensive and not well exploited by most accelerator applications as by construction these applications minimize false-sharing. This paper investigates the possibility of an alternative tradeoff of cache-coherency and cache-maintenance block size for many-core accelerators, by decoupling coherency-block and cache-lines sizes. Interests, advantages and difficulties are presented and discussed in this paper. Then we also discuss needs of software and hardware modifications in prototypes and the capability of such prototypes to evaluate different coherenceblock sizes.

We present an approach for implementing in-place FFTs on cores fitted with SIMD units and non-temporal load-store units. Loading the input samples with SIMD instructions decimates them in time across the SIMD lanes. A classic FFT implementation is extended to operate on SIMD data rather than scalar data and computes the sub-transforms concurrently. This enables efficient exploitation of the SIMD arithmetic and memory access instructions while involving little SIMD lane shuffling. A last FFT stage then recombines in-place the sub-transforms results to produce the output. We illustrate this approach on a Cooley- Tukey radix-4 decimated-in-frequency FFT implementation, which also integrates the two inner loop collapsing optimization of the TI C6x DSP _fft32×32 code that enables software pipelining and the Burrus technique for using bit-reversal in high-radix FFT implementations. Performance evaluations are performed on the Kalray KV3 core, which implements a 64-bit vector-scalar VLIW architecture with level-l cache bypass load instructions.

This chapter presents the third-generation Massively Parallel Processor Array (MPPA) processor, manufactured in 16FFC CMOS technology, whose many-core architecture has significantly improved upon the previous ones in the areas of performance, programmability, functional safety and cyber-security. It discusses many-core architectures and their limitations with regard to intelligent system requirements. The chapter presents the main features of the third-generation MPPA architecture and processor. It introduces the MPPA3 application software environments. The structuring of the MPPA3 architecture into a collection of compute units, each comparable to an embedded multi-core processor, is the main feature that enables the consolidation of application partitions operating at different levels of functional safety and cyber-security, on a single processor. High-integrity computing on the MPPA3 processor refers to applications that execute in a physically isolated domain of the processor, whose functions are developed under model-based design and must meet hard real-time constraints.

Summary form only given, as follows. A complete record of the panel discussion was not made available for publication as part of the conference proceedings. Edge computing applications such as autonomous driving systems (ADS) and 5G radio access network (RAN) require significant computing capabilities and predictable response times, while being constrained by size, weight and power (SWaP). Such applications significantly benefit from computing platforms based on manycore processors. We first expose the differences between multi-core architectures and many-core architectures, currently mainly represented by GPGPU processors. Then, by using the MPPA3 processor from Kalray as an illustration, we present some of the challenges and the choices involved by engineering an edge processing computing platform based on a manycore architecture. On the local architecture, energy efficiency and time predictability can be leveraged from a Fisher-style VLIW architecture. Accelerating deep learning inference is achieved by tightly coupling a tensor coprocessor. On the global architecture, the cache coherence domains are preferably localized to the compute units. These compute units are connected by a network-on-chip capable of multi-casting, where deadlock-free routing requires some care. The computing platform is completed by providing standard and open programming environments. Among these, OpenCL, Open VX and OpenMP appear as the most relevant for compute-intensive edge applications, once these environments are enabled to efficiently exploit the compute unit local memories of the manycore architecture.

Deep neural networks need to be compressed due to their high memory requirements and computational complexity. Numerous compression methods have been proposed to solve this issue, but we still do not fully understand how the compression error will impact the neural networks. We take inspiration from the rate distortion theory to propose a new distortion function which measures the gap between the Bayes risk of a classifier before and after the compression. Since this distortion is not tractable, we derive a theoretical closed-form approximation when the last fully connected layer of a deep neural network is compressed with a uniform quantizer. This approximation provides insight into the relationship between the accuracy loss and some key characteristics of the neural network. Numerical simulations show that the approximation is reasonably accurate.

We study the implementation of data-flow applications on multi-core processor with on-chip shared multi-banked memory. Specifically, we consider the Kalray MPPA2 processor and three applications coded using the industrial toolchain SCADE Suite. We focus on the runtime environment assuming global static scheduling, time-triggered and non-preemptive execution of tasks. Our contributions include (i) a technique to implement SCADE applications compliant with execution models inspired by PREMs (PRe-dictable Execution Models), (ii) an exhaustive comparison of three execution models with and without isolation, and finally (iii) guidelines for predictable implementation of a data-flow application on multi-core processors with shared on-chip memory.

We consider hard real-time applications running on many-core processor containing several clusters of cores linked by a Network-on-Chip (NoC). Communications are done via shared memory within a cluster and through the NoC for inter-cluster communication. We adopt the time-triggered paradigm, which is well-suited for hard real-time applications, and we consider data-flow applications, where communications are explicit. We extend the AER (Acquisition/Execution/Restitution) execution model to account for all delays and interferences linked to communications, including the interference between the NoC interface and the memory. Indeed, for NoC communications, data is first read from the initiator's local memory, then sent over the NoC, and finally written to the local memory of the target cluster. Read and write accesses to transfer data between local memories may interfere with shared-memory communication inside a cluster, and, as far as we know, previous work did not take these interferences into account. Building on previous work on deterministic network calculus and shared memory interference analysis, our method computes a static, time-triggered schedule for an application mapped on several clusters. This schedule guarantees that deadlines are met, and therefore provides a safe upper bound to the global worst-case response time.

This article focuses on the trends, opportunities, and challenges of novel arithmetic for deep neural network (DNN) signal processing, with particular reference to assisted- and autonomous driving applications. Due to strict constraints in terms of the latency, dependability, and security of autonomous driving, machine perception (i.e., detection and decision tasks) based on DNNs cannot be implemented by relying on remote cloud access. These tasks must be performed in real time in embedded systems on board the vehicle, particularly for the inference phase (considering the use of DNNs pretrained during an offline step). When developing a DNN computing platform, the choice of the computing arithmetic matters. Moreover, functional safe applications, such as autonomous driving, impose severe constraints on the effect that signal processing accuracy has on the final rate of wrong detection/decisions. Hence, after reviewing the different choices and tradeoffs concerning arithmetic, both in academia and industry, we highlight the issues in implementing DNN accelerators to achieve accurate and lowcomplexity processing of automotive sensor signals (the latter coming from diverse sources, such as cameras, radar, lidar, and ultrasonics). The focus is on both general-purpose operations massively used in DNNs, such as multiplying, accumulating, and comparing, and on specific functions, including, for example, sigmoid or hyperbolic tangents used for neuron activation.

The Kalray MPPA2-256 processor integrates 256 processing cores and 32 management cores on a chip. These cores are grouped into clusters and clusters are connected by a high-performance network on chip (NoC). This NoC provides hardware mechanisms (ingress traffic limiters) that can be configured to offer service guarantees. This paper introduces a network calculus formulation, designed to configure the NoC traffic limiters, that also computes guarantee upper bounds on the NoC traversal latencies. This network calculus formulation accounts for the traffic shaping performed by the NoC links, and can be solved using linear programming. This paper then shows how existing network calculus approaches (the Separated Flow Analysis - SFA ; the Total Flow Analysis - TFA ; the Linear Programming approach - LP) can be adapted to analyze this NoC. The delay bounds obtained by the four approaches are then compared on two case studies: a small configuration coming from a previous study, and a realistic configuration with 128 or 256 flows. From these cases studies, it appears that modeling the shaping introduced by NoC links is of major importance to get accurate bounds. And when all packets have the same size, modeling it reduces the bound by 20%-25% on average. (C) 2020 Elsevier B.V. All rights reserved.

REE N°2/2019 Z59 DOSSIER 1 L'ELECTRONIQUE DU FUTUR Benoit Dupont de Dinechin Directeur de la technologie a Kalray Processeurs massivement paralleles pour systemes intelligents L’electronique numerique integree est confrontee aux limites de densite de puissance dissipee par les proces- seurs performants. Lorsque ces processeurs sont impliques dans la realisation de systemes intelligents, ils doivent de plus repondre en temps reel et garantir une surete de fonctionnement. La societe Kalray realise des proces- seurs pour systemes intelligents bases sur une architecture pluri-cœurs appelee MPPA. Introduction Face au ralentissement des progres de l’electronique numerique integree, les architectures de calculateurs evo- luent vers l’integration d’un nombre croissant de cœurs de traitement, afin de pouvoir poursuivre la progression des performances et de l’efficacite ener- getique demandees par les nouvelles applications. La generation des circuits logiques CMOS en geometrie 65 nm, deployee aux alentours de 2006, ma- terialise le niveau d’integration a partir duquel les lois d’echelle de Dennard ont cesse de s’appliquer