With shrinking device geometries, the device-to-device (D2D) variability of emerging memory devices, affect the performance of deep neural networks (DNN) and restrict the use of standalone devices to be used as Analog in-Memory Computing (AiMC) accelerators. For FeFET based AiMC designs, such effects can be controlled by using resistors or capacitors connected in series with FeFETs. However, in both cases, the effect of variability will again start dominating when the programming window shrinks further due to scaling towards smaller nanometer dimensions. Additionally, while resistance based solutions will add to the existing IR drop issues within the macro, capacitor based approach avoids such a case. In this paper, we evaluate a FeFET and capacitor based AiMC accelerator and study how finetuning cell operating parameters along with design modifications of the AiMC macro, can help in achieving a variability aware AiMC design. Using a 22-nm CMOS PDK augmented with FeFET model based on measured data, evaluation on a DNN with CIFAR-10 benchmark, demonstrated that following our approach, the implemented DNN achieves an accuracy of ∼85% which is an improvement of ∼13% over an untuned AiMC macro.

DIgital-ANAlog (DIANA), a heterogeneous multi-core accelerator, combines a reduced instruction set computer - five (RISC-V) host processor with an analog in-memory computing (AIMC) artificial intelligence (AI) accelerator and a digital reconfigurable deep neural network (DNN) accelerator in a single system-on-chip (SoC) to support a wide variety of neural network (NN) workloads. AIMC cores can bring extreme computational parallelism and efficiency at the expense of accuracy and dataflow flexibility. Digital AI co-processors, on the other hand, guarantee accuracy through deterministic compute, but cannot achieve the same computational density and efficiency. DIANA exploits this fundamental tradeoff by integrating both types of cores in a shared and optimized memory system, to enable seamless execution of the workloads on the parallel cores. The system’s performance benefits further from pipelined parallel execution across both accelerator cores and enhanced AIMC spatial unrolling techniques, leading to drastically reduced execution latency and reduced memory footprints. The design has been implemented in a 22-nm technology and achieves peak efficiencies of 600 TOP/s/W for the AIMC core (I/W/O: 7/1.5/6 bit) and 14 TOP/s/W (I/W/O: 8/8/8 bit) for the digital accelerator, respectively. End-to-end performance evaluation of CIFAR-10 and ImageNet classification workloads is carried out on the chip, reporting 7.02 and 5.56 TOP/s/W, respectively, at the system level.

GPGPU execution analysis has always been tied to closed-source, proprietary benchmarking tools that provide high-level, non-exhaustive, and/or statistical information, preventing a thorough understanding of bottlenecks and optimization possibilities. Open-source hardware platforms offer opportunities to overcome such limits and co-optimize the full hardware-mapping-algorithm compute stack. Yet, so far, this has remained under-explored. In this work, we exploit micro-architecture parameter analysis to develop a hardware-aware, runtime mapping technique for OpenCL kernels on the open Vortex RISC-V GPGPU. Our method is based on trace observations and targets optimal hardware resource utilization to achieve superior performance and flexibility compared to hardware-agnostic mapping approaches. The technique was validated on different architectural GPU configurations across several OpenCL kernels. Overall, our approach significantly enhances the performance of the open-source Vortex GPGPU, contributing to unlocking its potential and usability.

We report on the first demonstration of multilevel multiply accumulation operations in 2TlC cell arrays based on amorphous IGZO TFTs for efficient analog in memory compute (AiMC) implementation. Device designs for the read and write transistors to meet the target specifications are discussed and implemented. Multilevel operations are realized thanks to the long retention time enabled by the ultra-low offcurrent (<1.5× 1$0^{-19}$A/$\mu$m) of the a-IGZO TFTs.

Deep neural networks (DNNs) are widely used in applications like image recognition for high prediction accuracy. Energy efficiency, flexibility, and area efficiency are critical parameters to bring DNN capabilities to the edge. This work presents an SRAM-based digital in-memory computing matrix-vector multiplier (DiMe MVM) that provides flexibility in terms of operand bit width and efficiently accommodates different matrix sizes. Measurement results on the 28 nm CMOS chip illustrate energy and throughput benefits over other state-of-the-art digital implementations.

Energy-efficient matrix-vector multiplications (MVMs) are key to bringing neural network (NN) inference to edge devices. This has led to a wide range of state-of-the-art MVM acceleration chips, which fall into two categories: 1) Digital NN accelerators [1]–[2], constituting widely parallel multiply-accumulate (MAC) arrays at medium (typically 4-8b) precision. 2) Analog in-memory compute (AiMC) NN accelerators [3]–[4], which enable much higher energy efficiencies and throughput per unit area at the cost of a reduced computational precision, reduced dataflow flexibility, and resulting reduced mapping efficiency for some layer configurations. Neither of these approaches dominates the other, as it depends on the layer type which approach is the optimal. The ideal processor would enable exploiting both digital and AiMC NN acceleration concepts and select the best accelerator depending on the layer characteristics. Consequently, this work presents DIANA, a low-power NN processing SoC, comprising a precision-scalable digital NN accelerator, an AiMC core, an optimized shared-memory subsystem and a RISC-V host processor to achieve SOTA end-to-end inference at the edge. This SoC includes innovations in: a) its 16x16 digital NN core with flexible dataflow for fully connected and high-precision CONV layer execution, b) its 1152x512 AiMC core with SIMD digital post-processing and support for output unrolling for improving array utilization, and c) a shared memory system supporting efficient layer-fused execution schedules, controlled by the RISC-V. This allows simultaneous execution of subsequent layers across the digital and analog cores, assigning high-precision layers and layers with limited AiMC utilization (e.g. FC layers and layers with low channel count) to the digital core, and all other intermediate layers to the AiMC core. A top-level overview of the designed system and its highlights is depicted in Fig. 15.6.1.

Recent years have seen a growing trend of deploying deep neural network-based applications on edge devices. Many of these applications, such as biometric identification, activity tracking, user preference learning, etc., require fine-tuning of the trained networks for user personalization. One way to prepare these models to handle new, unseen tasks, is to pretrain them on a distribution of known tasks. This observation has led to increasing research into meta-learning based few-shot learning techniques. However, basic meta-learning approaches do not account for the limited memory and computational resources during on-chip training. We propose a modified metalearning algorithm that enables quantized fine-tuning to optimally condition the models for on-chip few shot learning. The modification involves the inclusion of target hardware constraints upfront in the meta-learning process. Block floating point datatypes with low precision mantissa bits are utilized in the forward and backward passes, to allow hardware-friendly adaptation. Experiments show that our algorithm provides better initializations than conventional algorithms, more suitable for efficient quantized fine-tuning. This allows the few-shot learner to achieve better convergence, in terms of accuracy and speed. Extensive experiments are also performed to analyze the impact of initialization on quantized fine-tuning and further corroborate the benefits of our method.

The size and parameter optimization for the 5-nm carbon nanotube field effect transistor (CNFET) static random access memory (SRAM) cell was presented in Part I of this article. Based on that work, we propose a carbon nanotube (CNT) SRAM array composed of the schematically optimized CNFET SRAM and CNT interconnects. We consider the interconnects inside the CNFET SRAM cell composed of metallic single-wall CNT (M-SWCNT) bundles to represent the metal layers 0 and 1 (M0 and M1). We investigate the layout structure of CNFET SRAM cell considering CNFET devices, M-SWCNT interconnects, and metal electrode Palladium with CNT (Pd-CNT) contacts. Two versions of cell layout designs are explored and compared in terms of performance, stability, and power efficiency. Furthermore, we implement a 16 Kbit SRAM array composed of the proposed CNFET SRAM cells, multiwall CNT (MWCNTs) inter-cell interconnects and Pd-CNT contacts. Such an array shows significant advantages, with the read and write overall energy-delay product (EDP), static power consumption, and core area of $0.28\times $ , $0.52\times $ , and $0.76\times $ respectively to 7-nm FinFET-SRAM array with copper interconnects, whereas the read and write static noise margins are 6% and 12% respectively larger than the FinFET counterpart.

Buried power rail (BPR) and back-side power delivery grid have been proposed as solutions to scaling challenges that arise beyond the 5-nm technology node, mainly to lower IR drop and further shrink area. This article demonstrates a holistic evaluation of this technology and its variants at the microprocessor level. This is carried out by taking an Arm Cortex-A53 design through the standard-VLSI physical design implementation flow on Imec’s iN6 node, equivalent to the industry 3-nm technology node, which features the buried power technology. The power, performance, area, on-chip IR drop, and off-chip voltage droop metrics are benchmarked, and implications on power gating are explored. An extensive Design-Technology-Co-Optimization (DTCO) study of the back-side power grid is presented to enhance the decoupling capacitance by sweeping associated technology parameters showcasing further optimization opportunities in manufacturing. The conclusions of this work highlight that the front-side (FS) power delivery network (PDN) with buried rails achieves a 25% lower on-chip IR drop and 17% lower off-chip voltage droop (power supply noise) resulting in 21% lower guard band voltage. On the other hand, the back-side power grid with BPRs achieves 85% lower on-chip IR drop and 30% off-chip voltage droop resulting in 60% lower guard band voltage. In addition, the impact of BPRs, and back-side power grids on power gated designs are evaluated.

This tutorial focuses on memory elements and analog/digital (A/D) interfaces used in mixed-signal accelerators for deep neural networks (DNNs) in machine learning (ML) applications. These very dedicated systems exploit analog in-memory computation (AiMC) of weights and input activations to accelerate the DNN algorithm. The co-optimization of the memory cell storing the weights with the peripheral circuits is mandatory for improving the performance metrics of the accelerator. In this tutorial, four memory devices for AiMC are reported and analyzed with their computation scheme, including the digital-to-analog converter (DAC). Moreover, we review analog-to-digital converters (ADCs) for the quantization of the AiMC results, focusing on the design trade-offs of the different topologies given by the context.

Analog in Memory Computing (AiMC) based neural network acceleration is a promising solution to increase the energy efficiency of deep neural networks deployment. However, the quantization requirements of these analog systems are not compatible with state-of-the-art neural network quantization techniques. Indeed, while the quantization of the weights and activations is considered by modern deep neural network quantization techniques, AiMC accelerators also impose the quantization of each Matrix Vector Multiplication (MVM) result. In most demonstrated AiMC implementations, the quantization range of MVM results is considered a fixed parameter of the accelerator. This work demonstrates that dynamic control over this quantization range is possible but also desirable for analog neural networks acceleration. An AiMC compatible quantization flow coupled with a hardware aware quantization range driving technique is introduced to fully exploit these dynamic ranges. Using CIFAR-10 and ImageNet as benchmarks, the proposed solution results in networks that are both more accurate and more robust to the inherent vulnerability of analog circuits than fixed quantization range based approaches.

Analog In-Memory Compute (AIMC) arrays can store weights and perform matrix-vector multiplication operations for Deep Convolutional Neural Networks (CNNs). A number of recent efforts have integrated AIMC arrays into hybrid digital-analog accelerators in a multi-layer parallel manner to achieve energy efficiency and high throughput. Multi-layer parallelism on large-scale tile-based architectures need efficient mapping support at the processing element (PE)-level ( e.g. , digital or analog processing elements) and tile-level. To find the most efficient architectures, fast and accurate design space exploration (DSE) support is required. In this paper, a novel DSE framework, AERO, is presented to characterize a CNN inference workload executing on hybrid tile-based architectures that supports multi-layer parallelism. Three characteristics can be seen in our DSE framework: (1) It presents a hierarchical Tile/PE-level mapping exploration strategy including inter-layer interaction, and allowing layer fusion/splitting configurations for PE-level mapping optimization. (2) It unlocks different Performance, Power and Area (PPA) exploration points under both sufficient and limited resource constraints, while limited resource case is not considered in prior works of multi-layer parallel architectures. The impact of weight loading and weight stationary mapping are analyzed for better insights into hybrid tile-based architectures. (3) It incorporates a detailed PPA model that supports a broad range of hybrid digital and analog units in a tile. Experimental case-studies are performed for realistic and relevant benchmarks such as MLP, CNNs (Lenet-5, Resnet-18,-34,-50 and −101).

In this article, we propose a carbon nanotube (CNT) field-effect transistor (CNFET)-based static random access memory (SRAM) design at the 5-nm technology node that is optimized based on the tradeoff between performance, stability, and power efficiency. In addition to size optimization, physical model parameters including CNT density, CNT diameter, and CNFET flat band voltage are evaluated and optimized for CNFET SRAM performance improvement. Optimized CNFET SRAM is compared with state-of-the-art 7-nm FinFET SRAM cell based on Arizona State University [ASAP 7-nm FinFET predictive technology models (PTM)] library. We find that the read, write EDPs, and static power of the proposed CNFET SRAM cell are improved by 67.6%, 71.5%, and 43.6%, respectively, compared with the FinFET SRAM cell, with slightly better stability. CNT interconnects both inside and in-between CNFET SRAM cells are considered to compose an all-carbon-based SRAM (ACS) array which will be discussed in the Part II of this article. A 7-nm FinFET SRAM cell with copper interconnects is implemented and used for comparison.

This paper presents a tiny charge injection-Successive Approximation (ci-SAR) A/D converter (ADC) to be integrated at the periphery of analog Matrix Vector Multiplication (MVM) accelerators for Deep Neural Network (DNN) inference. Derived from the ci-SAR ADC, this converter exploits a single charge injecting cell to minimize area and energy consumption. The ADC exhibits a signal-to-noise and distortion ratio of 30.5 dB, at 5 bits of nominal resolution. The energy per conversion is 86 fJ, running at 34 MS/s, with a silicon area of $75 \mu m ^{2}$, in 22 nm technology node. From the results of our analytical framework, an SRAM-based Analog in-Memory Compute (AiMC) array, including the proposed ADC at 5 bits of resolution, can achieve an energy efficiency of 1650 TOPs/W.

Large weight variations cause significant degradation in Deep Neural Networks (DNNs) accuracy in machine learning (ML) context. Indium-Gallium-Zinc-Oxide (IGZO) DRAM compute cell is a promising option for Analog in-Memory Computing (AiMC) accelerators, but its applicability requires the compensation of large variations affecting the voltage threshold of the readout device. This paper proposes a write-verify scheme for IGZO-based AiMC accelerator, designed in 22-nm technology, based on a compensation loop operating on the analog weight value stored in the IGZO DRAM cell. After the compensation routine, the IGZO I ON normalized variation drops from $\pm \mathbf{2 7} \%$ to $\pm 3 \%$ in simulation. With sufficiently large weight reuse, the additional energy spent for the compensation of the entire arrays becomes negligible, recovering the 2000 TOPS/W baseline performance of an ideal IGZO array without the write-verify.

Analog in memory computing (AiMC) is a promising hardware solution to efficiently perform inference with deep neural networks (DNNs). Similar to digital DNN accelerators, AiMC systems benefit from aggressively quantized DNNs. In addition, AiMC systems also suffer from noise on activations and weights. Training strategies to condition DNNs against weight noise can increase the efficiency of AiMC systems by enabling the use of more compact but more noisy weight memory devices. In this work, we utilize noise-aware training and introduce gradual noise training and network width scaling to increase the tolerance of DNNs against weight noise. Our results show that noise-aware training and gradual noise training drastically lowers the impact of weight noise without changing the network size. By utilizing network width scaling, the weight noise tolerance is increased even more with the penalty of more network parameters.

Deep neural network (DNN) inference can be performed efficiently with analog in memory computing (AiMC). MRAM is an attractive solution to implement the DNN weights due to its non-volatility and scalability. However, accurate inference requires memories with multi-level conductance values. while MRAM is binary. In this work, we propose and demonstrate a multi-level SOT-MRAM device concept by placing multiple MTJ pillars between a single SOT track and common top electrode. Selective level programming is achieved by smartly using a pillar-position dependent VCMA-assist effect. Three DNN algorithm-driven technology requirements are derived: number of conductance levels, bit-error rate and conductance variation. This work demonstrates that multi-pillar SOT-MRAM meets all derived specifications, making it a promising candidate as memory device for accurate analog in-memory DNN inference.

Deep learning constitutes the state-of-the-art in machine learning, from data mining to computer vision and natural language processing. Energy-efficient matrix-vector multiplications (MVMs) [1] are key to bringing these capabilities to the edge, for both convolutional and recurrent neural networks. Since low precision multiplications with a limited level of noise can achieve relatively high accuracies [2], a compelling opportunity arises for mixed-signal hardware implementations to exploit in-memory computing paradigms [3– 6]. In this work, a matrix-vector multiplier is presented operating on ternary weights, 7-bit input (6-bit magnitude and 1-bit sign), and 6-bit output activations. Notable improvements in energy efficiency and peak compute throughput per unit area are exhibited by measurements on a test chip based on the GLOBALFOUNDRIES® 22FDX® FD-SOI platform, thanks to a novel SRAM-based compute cell with low “on” current, pulse-width encoded input activations, a clamp-free summation line, and efficient amortization of the ADC and DAC energies.

Extremely energy-efficient convolutional neural network inference (CNN) is recently enabled by analog in-memory compute (AiMC). The integration of AiMC in a primarily digital inference system brings new challenges ranging from device specifications to defining novel system architectures. A novel framework to evaluate the impact of AiMC array at the system level is presented. The framework is used to model a SRAM-based 1024 × 512 prototype AiMC array, capable of energy efficiency of upto 675 TMACs/W. The proposed framework allows modelling different compute cells, array dimensions, operating voltage, and activation buffer energy models which can be used to determine the overall energy efficiency for various CNN workloads.