Encoding, model, and architecture: systematic optimization for spiking neural network in FPGAs

ABSTRACTSpiking neural network (SNN) has drawn research interests as it mimics dynamic activities of human brain and has the potential to perform real-time cognitive tasks. However, latency, throughput and flexibility of existing hardware implemented SNNs are limited. The conventional rate coding is inefficient in terms of accuracy and latency. Oversimplified SNN models adopted by neuromorphic hardware discard characteristics such as neuron dynamics and filter effects etc., which are critical for neural information processing. Recent research advancements show that the potential of SNN can be better utilized by moving beyond rate-based model and considering temporal information embedded in the spike sequences. However, these works employ complex biologically realistic SNN models, posing challenges to hardware complexity. Furthermore, most existing neuromorphic hardware are developed for specific SNN models, or aiming at replicating biological behaviors. There is a lack of general methodology for SNN design optimization. Novel hardware architecture and systematic optimization techniques are required for efficient FPGA implementation and support flexible SNN models. To address above issues, in this work we proposed a holistic optimization framework for encoder, model, and architecture design of FPGA based neuromorphic hardware. We present an efficient neural coding scheme and training algorithm, which can optimize encoder parameters to enable fast inference. A flexible and hardware-friendly model is proposed, in which SNNs are represented as a network of Infinite Impulse Response (IIR) filters. Finally, an end-to-end framework is developed to optimize and deploy FPGA implementation. Experimental results show our work achieves state-of-the-art accuracy in various classification tasks, and outperforms various platforms including CPU, GPU and dedicated neuromorphic processors in terms of latency and throughput.