Input Conditioned Subranging and Skewed Quantisation of MACs in IMC

In-Memory Computation(IMC) of Neural-Network(NN) inference is done by performing the Multiply-ACcumulate(MAC) operation in the analog domain. Parallelly digitising MAC voltages by fitting Analog to Digital Converters(ADCs) within dense memory-pitches is a fundamental challenge for IMC engines. IMC works thus far rely on clipping the MAC-PDF to reduce the dynamic range, reducing the per data-line(DL) ADC precision requirement. In this work, we show that the per-DL ADC precision can be reduced even further by focusing on quantising the input Conditioned MAC-PDF(CMPDF), which spans a sub-range in the total MAC-PDF. We demonstrate on hardware a technique to locate the CMPDF in one-shot by tracking its mean. We show that quantisation levels about the CMPDF mean can be skewed to only span the portion of CMPDF that yields positive ReLU inputs, provided MACs are implemented as complete sums. Compared to symmetrically spanning CMPDF, this requires 20% to 40% fewer references at iso-accuracy for the investigated neural network layers. Hardware measured results for Fully-Connected NN inference on MNIST yielded < 1% accuracy drop when MAC-voltages were quantised with 4 bit references about the CMPDF mean as compared to full-range 6 bit ADC sensing the MAC-voltages. MATLAB evaluation using 3.5 to 4.2 bit CMPDF quantisation for MNIST-FCNN, CIFAR-10 Resnet-20 and VGG-11 inference yielded < 1% accuracy drop as compared to full-range 6 to 7 bit MAC quantisation.