Analog in‐memory computing (AIMC) based on non‐volatile memories (NVMs) offers a transformative pathway toward ultra‐energy‐efficient deep learning, enabling high‐speed, low‐power inference with scalable performance. Recent progress has demonstrated floating‐point‐level accuracy through hardware‐aware training, highlighting AIMC's potential to rival conventional digital architectures. As NVM technologies for AIMC continue to advance, a systematic framework for defining device specification targets is critical for balancing inference accuracy and fabrication cost. Key device parameters, such as memory window, read and program noise, and conductance drift, directly influence inference fidelity across different time scales while simultaneously affecting manufacturability. Their strong interdependence complicates optimization, as tuning one parameter can relax or tighten constraints on others. To address this challenge, we propose a comprehensive methodology for mapping the multidimensional device specification space required to achieve floating‐point‐equivalent accuracy, using phase‐change memory (PCM) as a representative platform. Cross‐model evaluations on convolutional neural networks, recurrent neural networks, and transformer architectures reveal that device characteristics must be co‐optimized within this coupled landscape. The resulting analysis identifies families of specification sets that deliver comparable accuracy, enabling flexible, cost‐effective hardware design and paving the way for scalable, commercially viable AIMC deployment.

Analog in-memory computing is a next-generation computing paradigm that promises fast, parallel, and energy-efficient deep learning training and transfer learning (TL). However, achieving this promise has remained elusive due to a lack of suitable training algorithms. Analog memory devices exhibit asymmetric and non-linear switching behavior in addition to device-to-device variation, meaning that most, if not all, of the current off-the-shelf training algorithms cannot achieve good training outcomes. Also, recently introduced algorithms have enjoyed limited attention, as they require bi-directionally switching devices of unrealistically high symmetry and precision and are highly sensitive. A new algorithm chopped TTv2 (c-TTv2), has been introduced, which leverages the chopped technique to address many of the challenges mentioned above. In this paper, we assess the performance of the c-TTv2 algorithm for analog TL using a Swin-ViT model on a subset of the CIFAR100 dataset. We also investigate the robustness of our algorithm to changes in some device specifications, including weight transfer noise, symmetry point skew, and symmetry point variability

A Lite Bidirectional Encoder Representations from Transformers model is demonstrated on an analog inference chip fabricated at 14nm node with phase change memory. The 7.1 million unique analog weights shared across 12 layers are mapped to a single chip, accurately programmed into the conductance of 28.3 million devices, for this first analog hardware demonstration of a meaningfully large Transformer model. The implemented model achieved near iso-accuracy on the General Language Understanding Evaluation benchmark of seven tasks, despite the presence of weight-programming errors, hardware imperfections, readout noise, and error propagation. The average hardware accuracy was only 1.8% below that of the floating-point reference, with several tasks at full iso-accuracy. Careful fine-tuning of model weights using hardware-aware techniques contributes an average hardware accuracy improvement of 4.4%. Accuracy loss due to conductance drift - measured to be roughly 5% over 30 days - was reduced to less than 1% with a recalibration-based "drift compensation" technique.

A critical bottleneck for the training of large neural networks (NNs) is communication with off-chip memory. A promising mitigation effort consists of integrating crossbar arrays of analogue memories in the Back-End-Of-Line, to store the NN parameters and efficiently perform the required synaptic operations. The "Tiki-Taka" algorithm was developed to facilitate NN training in the presence of device nonidealities. However, so far, a resistive switching device exhibiting all the fundamental Tiki-Taka requirements, which are many programmable states, a centered symmetry point, and low programming noise, was not yet demonstrated. Here, a complementary metal-oxide semiconductor (CMOS)-compatible resistive random access memory (RRAM), showing more than 30 programmable states with low noise and a symmetry point with only 5% skew from the center, is presented for the first time. These results enable generalization of Tiki-Taka training from small fully connected networks to larger long-/short-term-memory types of NN.

Numerical computation is essential to many areas of artificial intelligence (AI), whose computing demands continue to grow dramatically, yet their continued scaling is jeopardized by the slowdown in Moore's law. Multi-function multi-way analog (MFMWA) technology, a computing architecture comprising arrays of memristors supporting in-memory computation of matrix operations, can offer tremendous improvements in computation and energy, but at the expense of inherent unpredictability and noise. We devise novel randomized algorithms tailored to MFMWA architectures that mitigate the detrimental impact of imperfect analog computations while realizing their potential benefits across various areas of AI, such as applications in computer vision. Through analysis, measurements from analog devices, and simulations of larger systems, we demonstrate orders of magnitude reduction in both computation and energy with accuracy similar to digital computers.

Analog memory presents a promising solution in the face of the growing demand for energy-efficient artificial intelligence (AI) at the edge. In this study, we demonstrate efficient deep neural network transfer learning utilizing hardware and algorithm co-optimization in an analog resistive random-access memory (ReRAM) array. For the first time, we illustrate that in open-loop deep neural network (DNN) transfer learning for image classification tasks, convergence rates can be accelerated by approximately 3.5 times through the utilization of co-optimized analog ReRAM hardware and the hardware-aware Tiki-Taka v2 (TTv2) algorithm. A simulation based on statistical 14 nm CMOS ReRAM array data provides insights into the performance of transfer learning on larger network workloads, exhibiting notable improvement over conventional training with random initialization. This study shows that analog DNN transfer learning using an optimized ReRAM array can achieve faster convergence with a smaller dataset compared to training from scratch, thus augmenting AI capability at the edge.

We developed a phase-change memory (PCM), with SiSbTe material, that showed state-independent resistance drift (v~0.04) at 65°C over the entire analog conductance range. We evaluated this PCM for In Memory Compute (IMC) applications simulating the performance of BERT model with the IBM Analog Hardware Acceleration Kit (AIHWKit). Drift and data retention are dependent on the amount of A-type dopant into SiSbTe materials. Finding a trade-off between the two is important to deliver a balanced material that can tackle IMC workload without losing in performance. The fabricated SiSbTe PCM devices maintain the BERT accuracy (<2% loss) for more than 7 days at 65°C and pass the data retention at 85 °C/48hrs demonstrating a great balance between the two metrics.

Given the high economic and environmental costs of using large vision or language models, analog in-memory accelerators present a promising solution for energy-efficient AI. While inference on analog accelerators has been studied recently, the training perspective is underexplored. Recent studies have shown that the "workhorse" of digital AI training - stochastic gradient descent (SGD) algorithm converges inexactly when applied to model training on non-ideal devices. This paper puts forth a theoretical foundation for gradient-based training on analog devices. We begin by characterizing the non-convergent issue of SGD, which is caused by the asymmetric updates on the analog devices. We then provide a lower bound of the asymptotic error to show that there is a fundamental performance limit of SGD-based analog training rather than an artifact of our analysis. To address this issue, we study a heuristic analog algorithm called Tiki-Taka that has recently exhibited superior empirical performance compared to SGD. We rigorously show its ability to converge to a critical point exactly and hence eliminate the asymptotic error. The simulations verify the correctness of the analyses.

Analog in-memory computing is a promising future technology for efficiently accelerating deep learning networks. While using in-memory computing to accelerate the inference phase has been studied extensively, accelerating the training phase has received less attention, despite its arguably much larger compute demand to accelerate. While some analog in-memory training algorithms have been suggested, they either invoke significant amount of auxiliary digital compute-accumulating the gradient in digital floating point precision, limiting the potential speed-up-or suffer from the need for near perfectly programming reference conductance values to establish an algorithmic zero point. Here, we propose two improved algorithms for in-memory training, that retain the same fast runtime complexity while resolving the requirement of a precise zero point. We further investigate the limits of the algorithms in terms of conductance noise, symmetry, retention, and endurance which narrow down possible device material choices adequate for fast and robust in-memory deep neural network training. Analog in-memory computing recent hardware implementations focused mainly on accelerating inference deployment. In this work, to improve the training process, the authors propose algorithms for supervised training of deep neural networks on analog in-memory AI accelerator hardware.

In-memory computing using analog non-volatile memory (NVM) devices can improve the speed and reduce the latency of deep neural network (DNN) inference. It has been recently shown that neuromorphic crossbar arrays, where each weight is implemented using analog conductance values of phase-change memory devices, achieve competitive accuracy and high power efficiency. However, due to the large amount of NVMs needed and the challenge for making analog NVM devices, these chips typically include some failed devices from fabrication or developed over time. We study the impact of these failed devices on the analog in-memory computing accuracy for various networks. We show that larger networks with fewer reused layers are more tolerable to failed devices. Devices stuck at high resistance states are more tolerable than devices stuck at low resistance states. To improve the robustness of DNNs to defective devices, we develop training methods that add noise and corrupt devices in the weight matrices during network training and show that this can increase the network accuracy in the presence of the failed devices. We also provide estimated maximum defective device tolerance of some common networks.

The need to repeatedly shuttle around synaptic weight values from memory to processing units has been a key source of energy inefficiency associated with hardware implementation of artificial neural networks. Analog in-memory computing (AIMC) with spatially instantiated synaptic weights holds high promise to overcome this challenge, by performing matrix-vector multiplications (MVMs) directly within the network weights stored on a chip to execute an inference workload. However, to achieve end-to-end improvements in latency and energy consumption, AIMC must be combined with on-chip digital operations and communication to move towards configurations in which a full inference workload is realized entirely on-chip. Moreover, it is highly desirable to achieve high MVM and inference accuracy without application-wise re-tuning of the chip. Here, we present a multi-core AIMC chip designed and fabricated in 14-nm complementary metal-oxide-semiconductor (CMOS) technology with backend-integrated phase-change memory (PCM). The fully-integrated chip features 64 256x256 AIMC cores interconnected via an on-chip communication network. It also implements the digital activation functions and processing involved in ResNet convolutional neural networks and long short-term memory (LSTM) networks. We demonstrate near software-equivalent inference accuracy with ResNet and LSTM networks while implementing all the computations associated with the weight layers and the activation functions on-chip. The chip can achieve a maximal throughput of 63.1 TOPS at an energy efficiency of 9.76 TOPS/W for 8-bit input/output matrix-vector multiplications.

Analog In-Memory Computing (AIMC) is a promising approach to reduce the latency and energy consumption of Deep Neural Network (DNN) inference and training. However, the noisy and non-linear device characteristics and the non-ideal peripheral circuitry in AIMC chips require adapting DNNs to be deployed on such hardware to achieve equivalent accuracy to digital computing. In this Tutorial, we provide a deep dive into how such adaptations can be achieved and evaluated using the recently released IBM Analog Hardware Acceleration Kit (AIHWKit), freely available at https://github.com/IBM/aihwkit. AIHWKit is a Python library that simulates inference and training of DNNs using AIMC. We present an in-depth description of the AIHWKit design, functionality, and best practices to properly perform inference and training. We also present an overview of the Analog AI Cloud Composer, a platform that provides the benefits of using the AIHWKit simulation in a fully managed cloud setting along with physical AIMC hardware access, freely available at https://aihw-composer.draco.res.ibm.com. Finally, we show examples of how users can expand and customize AIHWKit for their own needs. This Tutorial is accompanied by comprehensive Jupyter Notebook code examples that can be run using AIHWKit, which can be downloaded from https://github.com/IBM/aihwkit/tree/master/notebooks/tutorial.

Analog Non-Volatile Memory-based accelerators offer high-throughput and energy-efficient Multiply-Accumulate operations for the large Fully-Connected layers that dominate Transformer-based Large Language Models. We describe architectural, wafer-scale testing, chip-demo, and hardware-aware training efforts towards such accelerators, and quantify the unique raw-throughput and latency benefits of Fully-(rather than Partially-) Weight-Stationary systems.

Analog non-volatile memory (NVM)-based accelerators for Deep Neural Networks (DNNs) can achieve high-throughput and energy-efficient multiply-accumulate (MAC) operations by taking advantage of massively parallelized analog compute, implemented with Ohm's law and Kirchhoff's current law on arrays of resistive memory devices. Competitive end-to-end DNN accuracies can be obtained, provided that weights are accurately programmed onto NVM devices and MAC operations are sufficiently linear. In this paper, we report architectural and circuit advances for such Analog NVM-based accelerators. We describe a highly heterogeneous and programmable accelerator architecture for DNN inference that combines analog NVM memory-array “Tiles” for weight-stationary, energy-efficient MAC operations, together with heterogeneous special-function Compute-Cores for auxiliary digital computation. Massively parallel vectors of neuron-activation data are exchanged over short distances using a dense and efficient circuit-switched 2D mesh, enabling a wide range of DNN workloads, including CNNs, LSTMs, and Transformers. We also show a 14-nm inference chip consisting of multiple $\mathbf{512}\times \mathbf{512}$ arrays of Phase Change Memory (PCM) devices which implements multiple DNN benchmarks using such a circuit-switched 2D mesh.

Among the emerging approaches for deep learning acceleration, compute-in-memory (CIM) in crossbar arrays, in conjunction with optimized digital computation and communication, is attractive for achieving high execution speeds and energy efficiency. Analog phase-change memory (PCM) is particularly promising for this purpose. However, resistance typically drifts, which can degrade deep learning accuracy over time. Herein, we first discuss drift and noise mitigation by integrating projection liners into analog mushroom-type PCM devices, as well as tradeoffs with dynamic range. We then study their impact on inference accuracy for the Transformer-based language model BERT. We find that accuracy loss after extended drift can be minimal with an optimized mapping of weights to cells comprising two pairs of liner PCM devices of varying significance. Finally, we address the impact of drift on energy consumption during inference through a combination of drift, circuit, and architecture simulations. For a range of typical drift coefficients, we show that the peak vector-matrix multiplication (VMM) energy efficiency of a recently proposed heterogeneous CIM accelerator in 14 nm technology can increase by 3% to 15% over the course of one day to ten years. For convolutional neural network (CNN), long short-term memory (LSTM) and Transformer benchmarks, the increase in sustained energy efficiency remains below 10%, being greatest for models dominated by analog computation. Longer VMM integration times increase the energy impact of drift.

Analog in-memory computing-a promising approach for energy-efficient acceleration of deep learning workloads-computes matrix-vector multiplications but only approximately, due to nonidealities that often are non-deterministic or nonlinear. This can adversely impact the achievable inference accuracy. Here, we develop an hardware-aware retraining approach to systematically examine the accuracy of analog in-memory computing across multiple network topologies, and investigate sensitivity and robustness to a broad set of nonidealities. By introducing a realistic crossbar model, we improve significantly on earlier retraining approaches. We show that many larger-scale deep neural networks-including convnets, recurrent networks, and transformers-can in fact be successfully retrained to show iso-accuracy with the floating point implementation. Our results further suggest that nonidealities that add noise to the inputs or outputs, not the weights, have the largest impact on accuracy, and that recurrent networks are particularly robust to all nonidealities.

In-memory computing with resistive crossbar arrays has been suggested to accelerate deep-learning workloads in highly efficient manner. To unleash the full potential of in-memory computing, it is desirable to accelerate the training as well as inference for large deep neural networks (DNNs). In the past, specialized in-memory training algorithms have been proposed that not only accelerate the forward and backward passes, but also establish tricks to update the weight in-memory and in parallel. However, the state-of-the-art algorithm (Tiki-Taka version 2 (TTv2)) still requires near perfect offset correction and suffers from potential biases that might occur due to programming and estimation inaccuracies, as well as longer-term instabilities of the device materials. Here we propose and describe two new and improved algorithms for in-memory computing (Chopped-TTv2 (c-TTv2) and Analog Gradient Accumulation with Dynamic reference (AGAD)), that retain the same runtime complexity but correct for any remaining offsets using choppers. These algorithms greatly relax the device requirements and thus expanding the scope of possible materials potentially employed for such fast in-memory DNN training.

A fully silicon‐integrated restricted Boltzmann machine (RBM) with an event‐driven contrastive divergence (eCD) training algorithm is implemented using novel stochastic leaky integrate‐and‐fire (LIF) neuron circuits and six‐transistor/2‐PCM‐resistor (6T2R) synaptic unit cells on 90 nm CMOS technology. To elaborate, designed a bidirectional, asynchronous, and parallel pulse‐signaling scheme over an analog‐weighted phase‐change memory (PCM) synapse array to enable spike‐timing‐dependent plasticity (STDP) as a local weight update rule based on eCD is designed. Building upon the initial version of this work, significantly more experimental details are added, such as the on‐chip characterization results of LIF and backward‐LIF (BLIF) and stochasticity of our random walk circuitry. The experimental characterization of these on‐chip stochastic neuron circuits shows a reasonable symmetricity between LIF and BLIF as well as the necessary stochasticity for spiking RBM operation. Fully hardware‐based image classification recorded 93% on‐chip training accuracy from 100 handwritten MNIST digit images. In addition, we experimentally demonstrated the generative characteristics of the RBM by reconstructing partial patterns on hardware. As each synapse and neuron execute its computations in an asynchronous and fully parallel fashion, the chip can perform data‐intensive machine learning (ML) tasks in a power‐efficient manner and take advantage of the sparseness of spiking.

Analog non-volatile memory (NVM)-based accelerators for deep neural networks perform high-throughput and energy-efficient multiply-accumulate (MAC) operations (e.g., high TeraOPS/W) by taking advantage of massively parallelized analog MAC operations, implemented with Ohm’s law and Kirchhoff’s current law on array-matrices of resistive devices. While the wide-integer and floating-point operations offered by conventional digital CMOS computing are much more suitable than analog computing for conventional applications that require high accuracy and true reproducibility, deep neural networks can still provide competitive end-to-end results even with modest (e.g., 4-bit) precision in synaptic operations. In this paper, we describe a 14-nm inference chip, comprising multiple 512$\times$ 512 arrays of Phase Change Memory (PCM) devices, which can deliver software-equivalent inference accuracy for MNIST handwritten-digit recognition and recurrent LSTM benchmarks, by using compensation techniques to finesse analog-memory challenges such as conductance drift and noise. We also project accuracy for Natural Language Processing (NLP) tasks performed with a state-of-art large Transformer-based model, BERT, when mapped onto an extended version of this same fundamental chip architecture.