Neuromorphic computing hardware has emerged as a promising candidate for energy-efficient edge AI devices. By concurrently processing spatio-temporal data, these accelerators exhibit strong potential in hybrid model architectures. However, frequent variations in data fed into hardware induce significant dynamic power consumption, severely limiting the practicality of such hardware. Although exploiting the sparse nature of data enhances computational efficiency, unstructured sparsity in both activations and weights poses additional challenges, resulting in substantial inefficiencies in computation. To overcome these issues, we propose DyNeuro, a hybrid neuromorphic accelerator for both spiking neural network and artificial neural network paradigms. DyNeuro incorporates a Dynamic Valid Product Aggregator with sign-magnitude encoding to eliminate redundant multiplications and accumulations based on output sparsity, and also dynamically centralizes spatial data. In addition, a Sparsity Aware Processing Unit is designed to minimize the size of activated circuits, achieving up to 46% power savings. The General Attention-based Dynamic Manager further adapts to temporal data variations by more than 50% timestep pruning. Implemented using 28-nm technology, DyNeuro achieves a peak energy efficiency of 60.2 TOPS/W and 41.2 TSOPS/W with 8-bit weights. The inference accuracy on DVS-Gesture is 94.44%.

This paper presents a multi-core neuromorphic processor DualLearn that enables unified inference and training for dual SNN-ANN models. The architecture features reconfigurable computing units that natively support both SNN and ANN training algorithms, and a distributed multi-core design ensures scalable parallel processing with dynamic resource allocation between training and inference phases. An adaptive memory organization further enhances flexibility for diverse network structures. Fabricated in 28nm CMOS technology, the processor achieves training accuracies of 75.00% on DVS-CIFAR10 and 70.37% on CIFAR-100, with a training energy efficiency of 4.686pJ/SOP for SNN and 4.026pJ/FLOP for ANN models.

Abstract There is a growing necessity for edge training to adapt to dynamically changing environments. Neuromorphic computing represents a significant pathway for highly efficient intelligent computation in energy-constrained edges, but existing neuromorphic architectures lack the ability of directly training spiking neural networks based on backpropagation. We developed a multi-core neuromorphic architecture with Feedforward-Propagation, Back-Propagation, and Weight-Gradient engines in each core, supporting highly efficient parallel computing at both the engine and core levels, achieving 190% ~ 330% performance of Jetson Orin. It combines various data flows and sparse computation optimization by fully leveraging the sparsity in spiking neural network training, obtaining a high energy efficiency of 1.05TFLOPS/W@ FP16 @ 28 nm, 55 ~ 85% reduction of memory access compared to A100 GPU in the training. Additionally, we deployed the architecture on Field Programmable Gate Arrays, successfully demonstrating 20-core deep spiking network training and 5-worker federated learning. Our study develops the first multi-core neuromorphic architecture supporting direct training of spiking neural network, facilitating neuromorphic computing in edge-learnable applications.

Neuromorphic computing has endeavored a novel computing paradigm that entails a bio-inspired architecture to reproduce the remarkable functionalities of the human brain, such as massively parallel processing and extremely low-power consumption. However, those promising merits can be greatly canceled by the mismatched communication infrastructure in large-scale hardware implementation, in view of the vast degree of neural connectivity, the unstructured spike dataflow, and the unbalanced model workload assignment. In an effort to tackle those challenges, this work presents NeuroHexa, a network-on-chip (NoC) architecture intended for multi-core neuromorphic design. NeuroHexa adopts a customized intra-chip hexagonal topology, which can be further cascaded in 6 directions by either 2D or 3D chiplet integration. Designed in globally asynchronous, locally synchronous (GALS) methodology, a group of processing nodes can operate in independent work pace to further improve resource utilization. To satisfy the varied requirement of data reuse across the chip, NeuroHexa proposes a flexible multicast routing mechanism to best adapt to the model-defined dataflow. And under a specific congestion scenario, NeuroHexa can switch its routing algorithm between deterministic routing and fully adaptive routing modes. The presented NoC router is evaluated in 28nm CMOS, where we achieve the maximal throughput as 179.2Gbps, and the best energy efficiency as 4.872pJ/packet at the area overhead of 0.0226mm2.

Spiking neural networks (SNNs) have shown their competence in handling spatial-temporal event-based data with low energy consumption. Similar to conventional artificial neural networks (ANNs), SNNs are also vulnerable to gradient-based adversarial attacks, wherein gradients are calculated by spatial-temporal back-propagation (STBP) and surrogate gradients (SGs). However, the SGs may be invisible for an inference-only model as they do not influence the inference results, and current gradient-based attacks are ineffective for binary dynamic images captured by the dynamic vision sensor (DVS). While some approaches addressed the issue of invisible SGs through universal SGs, their SGs lack a correlation with the victim model, resulting in sub-optimal performance. Moreover, the imperceptibility of existing SNN-based binary attacks is still insufficient. In this paper, we introduce an innovative potential-dependent surrogate gradient (PDSG) method to establish a robust connection between the SG and the model, thereby enhancing the adaptability of adversarial attacks across various models with invisible SGs. Additionally, we propose the sparse dynamic attack (SDA) to effectively attack binary dynamic images. Utilizing a generation-reduction paradigm, SDA can fully optimize the sparsity of adversarial perturbations. Experimental results demonstrate that our PDSG and SDA outperform state-of-the-art SNN-based attacks across various models and datasets. Specifically, our PDSG achieves 100% attack success rate on ImageNet, and our SDA obtains 82% attack success rate by modifying only 0.24% of the pixels on CIFAR10DVS. The code is available at https://github.com/ryime/PDSG-SDA.

In order to achieve the brain-like advantages over conservative computers, previous neuromorphic researchers have stretched the hardware explorations of the hybrid artificial neural network (ANN) and spiking neural network (SNN) inference approaches, as well as the efficient bio-plausible and gradient-based SNN training mechanisms. However, a versatile accelerator for both ANN-SNN inference and training is little addressed. In this work, we introduce HyNITA, a neuromorphic processor that supports accelerating both inference and training tasks of hybrid ANN and SNN models. Regarding the similarity and distinction, a pair of working stages are distinguished and distributed to multiple simple cores. The accelerator optimizes the interchange dataflow in a scalable chip design, following a reconfigurable design methodology to integrate the involved equation calculations in the dynamic process of neurons. The evaluation results show it achieves an accuracy of 99.65% and 99.34% on training ANN MNIST and SNN N-MNIST datasets.

The physical unclonable function (PUF) serves as a security primitive of circuits, which is applicable to the embedded systems with lightweight authentication function. However, the modeling attack, which estimates the unknown CRPs by establishing the mathematical model of PUF, is a real threat to the PUF based cryptosystems. Subsequently, the anti-modeling-attack PUF becomes a research hotspot. The systematic design method of secure PUF is still an open issue, although some secure PUF schemes have been proposed based on the repeated trials. This work proposes a security improvement method of PUFs to enhance the anti-modeling-attack capability. The growth function and the Vapnik-Chervonenkis (VC) dimension of PUF are defined as the indicators of PUF security. The proposed method regards the improvement of PUF as an optimization problem, which aims to obtain a PUF scheme with the better security indicators. Guided by the indicators, the proposed method is able to specify the improvement sites of PUF and the techniques to be applied. In addition, three approaches are proposed to inspire the new security improvement techniques. An improved arbiter PUF and an improved array-based PUF are designed as the instances of the results from the proposed method. Both of the improved PUF schemes have the stronger security than the original schemes.

The fusion of convolutional neural network (CNN) and spiking neural network (SNN) harnesses their complementary strengths in edge video tasks. However, ongoing challenges remain in the low compatibility between two neural networks and the shortage of storage resources. In this brief, we propose a reconfigurable digital compute-in-memory (RDCIM) architecture featuring 2:4 structured weight pruning, tailored for the efficient fusion of CNN and SNN. Specifically, differential frame convolution (DFC) is adopted in place of conventional convolution. Therefore, the RDCIM fusion accelerator can operate in either DFC mode or SNN mode based on the sparsity of activations. First of all, fine-grained 2:4 structured sparsity enables compressed weight storage, thereby doubling the throughput. Secondly, a novel bitwise in-memory Booth multiplier is implemented to support both 4-bit and 8-bit activations, offering flexible computation. Thirdly, the high sparsity inherent in both differential frames and input spikes allows for the concurrent support of three sparse optimization strategies: coarse-grained zero skipping, medium-grained bank gating, and fine-grained precharge gating. Implemented by TSMC 28nm technology, our proposed architecture achieves the peak energy efficiency of 42.02 TOPS/W for DFC and 0.02 pJ/SOP for SNN.

Neuromorphic computing is attracting significant attention due to its bio-mimetic characteristics. Consequently, neuromorphic hardware platforms have emerged as innovative computing architectures for acceleration. However, the fixed nature of data flow and resources leads to considerable inefficiencies in storage and computation, thereby limiting both utilization efficiency and overall performance. This severely hinders the deployment of edge artificial intelligence (AI) models. To address these issues, we present a multi-core neuromorphic accelerator named CROSSCUT. This crossbar-based system supports both spiking neural network (SNN) and artificial neural network (ANN) paradigms and has a capacity of 256K neurons and 288M synapses. By leveraging the Neuron Package Mechanism (NPM) and Synapse Compress Mechanism (SCM), CROSSCUT can increase input data scale by 64 times and reduce wasted resources and computations by 46.7%, ensuring high compatibility with diverse network structures in machine learning models. Additionally, a Tree-Mesh hybrid network on chip (NoC) is constructed for inter-core communication. Implemented on Xilinx XCVU9P FPGA, CROSSCUT can achieve a peak performance of 431.9 GSOPS/s and 121.13 GSOPS/W energy efficiency. The inference accuracy on MNIST is 98.2%.

Spiking neural networks (SNNs) operating with asynchronous discrete events show higher energy efficiency with sparse computation. A popular approach for implementing deep SNNs is artificial neural network (ANN)-SNN conversion combining both efficient training of ANNs and efficient inference of SNNs. However, the accuracy loss is usually nonnegligible, especially under few time steps, which restricts the applications of SNN on latency-sensitive edge devices greatly. In this article, we first identify that such performance degradation stems from the misrepresentation of the negative or overflow residual membrane potential in SNNs. Inspired by this, we decompose the conversion error into three parts: quantization error, clipping error, and residual membrane potential representation error. With such insights, we propose a two-stage conversion algorithm to minimize those errors, respectively. In addition, we show that each stage achieves significant performance gains in a complementary manner. By evaluating on challenging datasets including CIFAR- 10, CIFAR- 100, and ImageNet, the proposed method demonstrates the state-of-the-art performance in terms of accuracy, latency, and energy preservation. Furthermore, our method is evaluated using a more challenging object detection task, revealing notable gains in regression performance under ultralow latency, when compared with existing spike-based detection algorithms. Codes will be available at: https://github.com/Windere/snn-cvt-dual-phase.

The application of artificial neural network (ANN) in video processing encounters significant challenges, including large data volumes, numerous linear operations, and high power consumption. The fusion of convolutional neural network (CNN) and spiking neural network (SNN) provides a dual benefit of achieving high accuracy while maintaining low power consumption. However, ongoing challenges remain in minimizing multiply-accumulate (MAC) operations and optimizing data movement. In this work, we propose a reconfigurable digital compute-in-memory (RDCIM) heterogeneous macro without the sense amplifier, tailored for the diverse computational demands of CNN and SNN. To improve energy efficiency, differential frame convolution (DFC) is adopted to mitigate the computational overhead. In addition, computational resources are functionally reused to accommodate four data flow types, supporting both DFC and SNN operations. Implemented by TSMC 28nm technology, the proposed RDCIM heterogeneous macro achieves the peak energy efficiency of 29.13 TOPS/W for DFC and 0.56 pJ/SOP for SNN, operating at a frequency of 284 MHz.

Spatio-Temporal backpropagation (STBP) is considered as a high-accuracy training method for braininspired spiking neural networks (SNN). However, due to different operand precisions, there are numerous challenges for hardware implementation during training and inference process. In this work, we propose a reconfigurable architecture for the computations of forward propagation, backward propagation, and parameter updates of fixed-point STBP. This architecture employs reconfigurable processing elements (PEs) that can operate in three modes, reducing hardware resource redundancy significantly. At the same time, we adopt dynamic clock techniques to reduce the timing redundancy caused by different bit-width computations and input spike sparsity, improving the throughput and energy efficiency of our architecture. Our prototype, implemented on the Xilinx Virtex platform, achieves a 62.3% average throughput improvement in the forward mode and attains a peak energy efficiency of 37.2 GOPS/W in the backward mode on the NMNIST dataset.

Brain-inspired spiking neural network (SNN) has recently attracted widespread interest owing to its event-driven nature and relatively low-power hardware for transmitting highly sparse binary spikes. To further improve energy efficiency, some matrix compression algorithms are used for weight storage. However, the weight sparsity of different layers varies greatly. For a multicore neuromorphic system, it is difficult for the same compression algorithm to adapt to all the layers of SNN model. In this work, we propose a weight density adaptation architecture with hybrid compression method for SNN, named Marmotini. It is a multicore heterogeneous design, including three types of cores to complete computation of different weight sparsity. Benefiting from the hybrid compression method, Marmotini minimizes the waste of neurons and weights as much as possible. Besides, for better flexibility, a reconfigurable core that can be configured to compute convolutional layer or fully connected layer is proposed. Implemented on Xilinx Kintex UltraScale XCKU115 field-programmable gate array (FPGA) board, Marmotini can operate at 150-MHz frequency, achieving 244.6-GSOP/s peak performance and 54.1-GSOP/W energy efficiency at 0% spike sparsity.

Convolutional neural networks require a huge amount of computation in video applications. For some specific tasks, such as surveillance, differential frame convolution reuses inter-frame data and significantly reduces multiplication and accumulation. However, there are still some challenges in improving energy efficiency of differential frame convolution on chips. Firstly, differential frame convolution brings additional on-chip storage for reusing inter-frame data. Secondly, in post-processing of differential frame convolution, there are more memory accessing and arithmetic logic operations. Therefore, sparse working mode is of vital importance for the post-processing. In response to these challenges, this work proposes an on-chip fusion storage architecture for energy-efficient differential frame convolution and a pixel-level pipeline data flow that supports the sparsity of features. The simulation of our accelerator implemented in 28nm CMOS can achieve energy efficiency by 3.09x compared with other state-of-the-art works

The resistive random access memory (RRAM) based in-memory computing (IMC) is an emerging architecture to address the challenge of the “memory wall” problem. The complementary resistive switch (CRS) cell connects two bipolar RRAM elements anti-serially to reduce the sneak current in the crossbar array. The CRS array is a generic computing platform, for the arbitrary logic functions can be implemented in it. The IMC CRS LUT consumes fewer CRS cells than the static CRS LUT. The CRS array has built-in polymorphic characteristics because the correct logic function cannot be distinguished based on the circuit layout. However, the logic state of every CRS cell can be readout after each operation. It helps the attacker to recover the correct function of the IMC CRS LUT. This work discusses the resistance analysis attack of the IMC LUT based on the CRS array. The proposed resistance analysis attack method is able to be applied to different computation styles based on the CRS array, such as the CRS IMPLY, CRS NOR-OR/NAND-AND, and so on. The attacker can recover the logic function of the LUT by tracing the states of CRS cells. Furthermore, an improved IMC CRS LUT method is proposed and discussed to enhance security. The simulation and analysis results show that the improved IMC CRS LUT can resist various attacks, and it maintains the polymorphic characteristics of the IMC CRS LUT. And the N-bit full adder circuit based on the improved IMC CRS NOR-OR LUTs achieves the best performance compared with the previous counterparts.

Computing-in-memory (CIM) has emerged as a promising architecture for energy-efficient edge neural network inference. While CIM designs have shown great potential to benefit from sparsity, there are still limitations in supporting output analog-digital conversion (ADC) sparsity in current designs due to redundant circuits and weight-refreshing overhead. In this paper, we propose a novel CIM design called OASIS (Output Activation SparsIty Support). OASIS employs a single-bit-line computation circuit to support output ADC sparsity, avoiding redundancy or overhead. Our design achieves up to 812.2 TOPS/W normalized energy efficiency at a typical 50% sparsity. We demonstrate its effectiveness by deploying all layers of quantized neural networks to the chip and evaluating them on the MNIST and CIFAR-10 datasets, achieving 99.20% and 84.33% accuracy, respectively.

In this paper, an efficient brain-inspired spiking convolutional neural network processor architecture is presented. This architecture supports sparse spiking convolutions with an event driven and highly parallel pipeline. A compressed address event representation is applied to record spikes, which represents the neuron firing results of adjacent 4 channels with a single address, reducing the storage and transmission overheads of spikes. We propose an adaptive delay asynchronous pipeline to address the performance degradation of pipelines caused by multi-layer adder trees. A FPGA prototype has been implemented on the Xilinx Virtex development board. The processor prototype achieves an efficiency of 28.3 GSOP/W and 0.035 nJ/SOP at equivalent clock frequency of 300 MHz to 400 MHz.

The spiking neural network (SNN) is suitable for the intelligent edge computing applications because of its low-power characteristic. This work designs a reconfigurable spiking neural network accelerator supporting the spatiotemporal backpropagation (STBP) training method. The reconfigurable architecture is proposed between the spatial convolution module and the temporal accumulation module of the SNN accelerator. A sparse zero-hopping mechanism is designed to exploit the input sparsity of SNN datasets, and a mask mechanism is introduced between the forward inference computation and the backward training computation to exploit the output sparsity. During the training process, the peak and average performances of the SNN accelerator are 5.57 TOPS and 4.96 TOPS respectively, the power consumption is 6.124 W and the energy efficiency is 0.81 TOPS/W. The peak and average performances of the SNN accelerator are 5.98 TOPS and 5.14 TOPS respectively, the power consumption is 6.943 W and the energy efficiency is 0.74 TOPS/W, during the inference process.

Radar emitter classification (REC) plays an important role in modern warfare. Traditional REC methods have difficulty identifying complex radar signals in the present day. Inspired by biology, spiking neural networks (SNNs) have gradually gained widespread attention due to their low power characteristics. Compared with convolutional neural networks (CNNs), SNNs are more suitable for application in the field of REC. The reason is that SNN can not only maintain higher accuracy in the presence of noise interference, but also reduce the power consumption of mobile devices. However, it is challenging to make full use of the input sparsity of radar emitter signals and the weight sparsity of pruned SNN models. In this paper, a 28-nm neuromorphic processor for REC named NeuroREC is proposed. It uses matrix compression algorithms to store sparse weights on chip, and designs corresponding spike detection circuits for this purpose. As a single-core design, we propose a ping-pong running mechanism to alleviate the imbalance between IO throughput and peak performance. Two SNN models for classifying RadioML2016.b and RadioML2018.a datasets are deployed on the chip, achieving competitive accuracy with only 8 timesteps, and demonstrating better robustness than CNN. Fabricated in 28-nm CMOS process, NeuroREC runs at frequencies ranging from 22.5MHz to 744MHz. Under specific sparsity conditions, it can reach an energy efficiency of 7.22TSOP/W for 8-bit weight.