Neuromorphic processors have garnered considerable interest in recent years for their potential in energy-efficient and high-speed computing. The Locally Competitive Algorithm (LCA) has been utilized for power efficient sparse coding on neuromorphic processors, including the first Loihi processor [4, 5]. With the Loihi 2 processor enabling custom neuron models and graded spike communication, more complex implementations of LCA are possible [9]. We present a new implementation of LCA designed for the Loihi 2 processor and perform an initial set of benchmarks comparing it to LCA on CPU and GPU devices. In these experiments LCA on Loihi 2 is orders of magnitude more efficient and faster for large sparsity penalties, while maintaining similar reconstruction quality. We find this performance improvement increases as the LCA parameters are tuned towards greater representation sparsity. Our study highlights the potential of neuromorphic processors, particularly Loihi 2, in enabling intelligent, autonomous, real-time processing on small robots, satellites where there are strict SWaP (small, lightweight, and low power) requirements. By demonstrating the superior performance of LCA on Loihi 2 compared to conventional computing device, our study suggests that Loihi 2 could be a valuable tool in advancing these types of applications. Overall, our study highlights the potential of neuromorphic processors for efficient and accurate data processing on resource-constrained devices.
Agent-based modeling (ABM) and simulation have emerged as important tools for studying emergent behaviors, especially in the context of swarming algorithms for robotic systems. Despite significant research in this area, there is a lack of standardized simulation environments, which hinders the development and deployment of real-world robotic swarms. To address this issue, we present Zespol, a modular, Python-based simulation environment that enables the development and testing of multi-agent control algorithms. Zespol provides a flexible and extensible sandbox for initial research, with the potential for scaling to real-world applications. We provide a topological overview of the system and detailed descriptions of its plug-and-play elements. We demonstrate the fidelity of Zespol in simulated and real-word robotics by replicating existing works highlighting the simulation to real gap with the milling behavior. We plan to leverage Zespol's plug-and-play feature for neuromorphic computing in swarming scenarios, which involves using the modules in Zespol to simulate the behavior of neurons and their connections as synapses. This will enable optimizing and studying the emergent behavior of swarm systems in complex environments. Our goal is to gain a better understanding of the interplay between environmental factors and neural-like computations in swarming systems.
Crossbar arrays based on emerging memristor technology offer significant power savings when running Artificial Neural Network (ANN) applications. An additional advantage of this technology is its non-volatility, which eliminates the need for costly loading of a complete ANN model from conventional memories, thus saving power and allowing immediate system availability. However, this advantage can arguably be seen as a disadvantage from a security perspective, since the ANN model remains on the crossbar indefinitely and is therefore vulnerable to theft. On the other hand, a disadvantage of the current memristor technology is its significantly limited write endurance. Considering these two constraints, we propose a highly secure and yet simple deployment method: "S3cure", which does not require encryption of individual stored values in the memristor, thus avoiding rewriting after each decryption and also rendering useless any unwanted extraction of the ANN model from the crossbar. In our proposed methodology, the ANN model is permuted before deployment by multiplying the model weight matrices with permutation matrices, and the inverse permutation vectors of these matrices become the access key needed at runtime to correctly exploit the model. The analysis and testing of our "S3cure" method reveals extremely difficult brute-force reverse engineering, proportional to the key size, and also shows a limited one-time performance overhead incurred by this implementation.
Rhythmic tasks that biological beings perform such as breathing, walking, and swimming, use specialized neural networks called central pattern generators (CPG). Spiking CPGs have already been implemented to control robot locomotion. This paper aims to take this concept further by designing and implementing a tunable bursting central pattern generator to control quadruped robots for the first time, to the best of our knowledge. Bursting CPGs allow for more granular control over the motion and speed of operation while retaining the low memory usage and latency capabilities of spiking CPGs. A bio-mimetic neuron model is chosen for this implementation which is highly optimized to run real-time on standard (Arduino microcontroller) and specialized (Intel Loihi) hardware. The Petoi bittle is chosen as the model hardware setup to showcase the efficiency of the proposed CPGs even in serial processing architectures. The CPG network is also realized in a completely asynchronous Loihi architecture to illustrate its versatility. The fully connected network running on CPG takes around 10 kilo bytes of memory (33% of Arduino capacity) to execute different modes of locomotion - walk, jump, trot, gallop, and crawl. Benchmarking results show that the bio-mimetic neurons take around 600 bytes (around 2%) more memory than Izhikevich neurons while being 0.02ms (around 14%) faster in isolated neuron testing.
Superconducting Optoelectronic Networks (SOENs) combine photonics and superconductors to instantiate computing systems that approach the fundamental limits of information processing in terms of speed and scalability. Overcoming the engineering challenges of integrating these technologies into one system has consistently been aided by using neuro-inspired architectures. SOENs are natively Spiking Neural Networks (SNNs) of loop neurons , which themselves are comprised of many subsequent dendrites organized into intricate morphologies. Therefore, SOENs at scale may embody highly complex superstructures that demand commensurate learning methods. To that end, we here propose a simple activity-based update rule (the arbor-update ) for dendritic learning that is found to successfully classify nine-pixel images with a single neuron. We test two amendments to the arbor-update on a winner-take-all (WTA) mutually inhibitory three-neuron network. Both elastic weight collision with dynamical boundaries and intermittent validation are found to improve convergence time and conditions. The arbor-update and its variants are scalable with SOENs and may even map to other systems. Importantly, all proposed learning methods are expected to be entirely implementable for on-chip learning in SOENs.
The Spiking Neural Network Computer Architecture (SpiNNaker) is a massively parallel computing system. As one of the most widespread platforms in the emerging field of neuromorphic engineering, SpiNNaker targets three main areas of research: computational neuroscience, computer science, and robotics. For the latter, the promise of low power computation and the potential for large scale simulations in real-time make SpiNNaker very attractive, especially for autonomous mobile applications. In this context, research groups typically use SpiNNaker's Ethernet interface to inject and extract sensori-motor signals into and from SpiNNaker. However, in cases where the data throughput increases, the on-board Ethernet port constitutes a critical bottleneck. Some groups have overcome such a problem to some extent by developing their own I/O interfaces to connect external devices --- sensors and actuators --- directly to SpiNNaker. However, such custom-developed interfaces allow only limited general applications, and they don't fully exploit the high-speed FPGA-based interconnect offered by the 48-chip SpiNNaker boards. In this manuscript, we present SPIF: a general-purpose FPGA-based SpiNNaker Peripheral Interface board that overcomes SpiNNaker's communication bottleneck by connecting to its native High-Speed Serial Links (HSSLs). We evaluate SPIF's performance in terms of event throughput and latency. Finally, we demonstrate SPIF's capabilities by feeding events from a high-resolution event camera into a real-time spiking convolutional neural network. The system can track the position of a small and extremely fast but salient stimulus in the visual field with negligibly low latency.
Innovation in the field of neuromorphic computing is characterized by long periods of slow, steady growth that are punctuated by periods of rapid discovery. In order to accelerate discovery in the field of neuromorphic computing and disrupt the process of slow and steady growth, we are proposing a reinforcement learning architecture that is able to automatically construct simple circuits. Rather than providing the reinforcement learning agent with specifications for existing devices, we ask the reinforcement learning agent to provide specifications for novel devices that, if fabricated, can solve a specified problem. We show that, by slightly changing the problem statement, we can cause the reinforcement learning agent to produce different device specifications. Ultimately, we expect that by generating many possible solutions, the reinforcement learning agent will accelerate innovation by stimulating insight into potential solutions for problems.
Spiking Neural Networks (SNNs) are becoming increasingly popular for their application in Edge Artificial Intelligence (Edge-AI) due to their sparse and low-latency computation. Among these networks, analog hardware SNNs are chosen for their ability to emulate complex dynamics in neurons and synapses, especially in integrated Metal Oxide Semiconductor (MOS) technology. They can form memories of external stimuli by modulating the strength of synaptic weights. In this context, binary weights are a common hardware design choice, due to their ease to program and store. The use of binary weights in SNNs worsens the bias introduced by the coding level of input stimuli (i.e. fraction of active input nodes), where the network activity is highly correlated to the number of excited neurons. In this paper, we present a Complementary Metal Oxide Semiconductor (CMOS) solution for the coding level bias, by proposing a novel circuit that employs synaptic normalisation at the neuron level. This circuit modifies the gain of the neuron depending on its input weights, with a small footprint and therefore high scalability.
Contextual grouping mechanisms in early visual cortex are thought to aid in perception of ambiguous textures, including partially obscured targets under real-world high dynamic range (HDR) luminance. Yet, deep neural networks struggle with naturalistic obscuration and illumination while requiring millions of neurons and power-hungry GPUs for processing. We hypothesized that contextual grouping mechanisms for edge and luminance processing may aid in localization of targets under natural obscuration and illumination. To address this issue, we developed a novel small (< 10,000 neurons) spiking neural network (SNN) that uses spike time correlations to leverage the combined luminance and orientation similarity of nearby image regions for image pre-processing, to support downstream deep neural network (DNN) target localization. The network has leaky integrate-and-fire neurons with current based (CuBa) synapses and is simulated using the Nengo LOIHI API, with potential application via Intel's LOIHI neuromorphic hardware. We collected 89 HDR images of a target dummy in a heavily wooded environment under varying occlusion and illumination. We used SNN preprocessing to adjust local image contrast based on the grouping mechanism, followed by a DNN classifier (Detectron2) to localize the target. Results show that a small SNN for image preprocessing can aid image segmentation and localization of occluded targets, marking an initial step towards more efficient and accurate target recognition under natural illumination and occlusion.
Neurons that fire multiple spikes on activation are commonly observed in biological systems, but the impact of their inclusion in neuromorphic systems has not been thoroughly analyzed. In this preliminary work, we begin an initial evaluation of the impact of multi-fire neurons on classification and control task performance. We show that networks with these neurons that are evolved for control tasks tend to perform worse than networks with single-fire neurons; however, we also show that networks with these multi-fire neurons perform significantly better than single-fire neurons when included in reservoir computing approaches for classification tasks.
Spiking Graph Convolutional Networks (SGCNs) are an emerging class of neural networks that operate on graph-structured data and leverage the advantages of both spiking neural networks and GCNs, making them ideal for graph representation learning tasks. However, deploying SGCNs on neuromorphic chips is challenging due to complex spiking dynamics, structured message computation, and limited chip resources, impeding their applications on terminal devices. In this work, we present an algorithm mapping strategy using a contemporary neuromorphic chip to efficiently deploy SGCNs and maximize chip resource utilization. Our mapping scheme leverages the spatio-temporal features of the chip and partitions computation graphs into small sub-graphs, enabling the reuse of computation resources in a temporal and spatial manner by a flexible scheme setting and corresponding scheduling. Simulation experiments show that our approach achieves favorable results on citation benchmark datasets by significantly reducing up to 79.8% memory usage and more than 60% computational requirements of SGCN deployment.
Analog memristive devices have the potential to merge computing and memory, support local learning, reach high densities, enable 3D stacking, and low energy consumption for neuromorphic computing applications. Yet, integration is challenged by the variability and complex nonlinear dynamics involved in the tuning of memristors, which is required in computing and memory applications. In this paper, we model the dynamic analog switching of memristive devices with an evolution-measurement state-space model. A physics-based compact model is extended to capture statistical distributions of the variability observed in memristors. Based on metal-oxide memristors and electronic measurement data, we applied Sequential-Monte Carlo (Particle Filter) techniques to infer underlying memristor model parameters. The result is validated by experimental data. Applying the calibrated statistical model, we propose an efficient adaptive pulse programming scheme, and performed a comparative analysis across widely applied write-and-verify techniques. We show improved programming control in the metrics of error, energy, and time in reaching target states.
In this paper, we highlight how computational properties of biological dendrites can be leveraged for neuromorphic applications. Specifically, we demonstrate analog silicon dendrites that support multiplication mediated by conductance-based input in an interception model inspired by the biological dragonfly. We also demonstrate spatiotemporal pattern recognition and direction selectivity using dendrites on the Loihi neuromorphic platform. These dendritic circuits can be assembled hierarchically as building blocks for classifying complex spatiotemporal patterns.
Neuromorphic computing has emerged as a promising solution to the power and memory requirements of embedded systems. However, mapping pre-trained neural networks onto diverse neuromorphic hardware architectures presents significant challenges. In this paper, we introduce GMap, a versatile, easy-to-use, and open-source python library designed for mapping neural networks onto any arbitrary hardware architecture. GMap adapts the simulated annealing approach, offering a probabilistic, meta-heuristic optimization for approximating the global optimum mapping. The algorithm takes architectural parameters of the hardware and network connectivity as input and provides a mapping solution as output. The library allows users to either map a pre-trained network onto a pre-existing chip or define custom hardware with respect to its constraints. GMap contributes to the neuromorphic field by enabling efficient deployment of neural networks on various hardware platforms and facilitating further research and collaboration in the neuromorphic community.
With the emergence of neuromorphic hardware as a promising low-power parallel computing platform, the need for tools that allow researchers and engineers to efficiently interact with such hardware is rapidly growing. Machine learning frameworks like Tensorflow, PyTorch and JAX have been instrumental for the success of machine learning in recent years as they enable seamless interaction with traditional machine learning accelerators such as GPUs and TPUs. In stark contrast, interfacing with neuromorphic hardware remains difficult since the aforementioned frameworks do not address the challenges associated with mapping neural network models and algorithms to physical hardware. In this paper, we review the various strategies employed throughout the neuromorphic computing community to tackle these challenges and categorize them according to their methodologies and implementation effort. This classification serves as a guideline for device engineers and software developers alike to enable them to choose the best-fit solution in regard of their demands and available resources. Finally, we provide a JAX-based proof-of-concept implementation of a compilation pipeline tailored to the needs of researchers in the early stages of device development, where parts of the computational graph can be mapped onto custom hardware via operations exposed through a C++ or Python interface. The code is available at https://github.com/PGI15/xbarax.
This work reports on new results and insights from the optimization of spiking neural networks developed for gamma-ray radiation anomaly detection. Our previous paper introduced the first known neuromorphic algorithm for this application, demonstrating promising results and insights into optimal hyperparameter selection - particularly in the choice of data input encodings. Since the first paper, we have tested the algorithms on new datasets to investigate transferability from one background radiation environment to another. We have also performed a new hyperparameter optimization experiment with this new dataset to investigate the impact of new radiation data formatting techniques, the inclusion or neuronal temporality, and neuron charge leakage. This paper provides an overview and discussion of the results from this study. Of note, we report that the inclusion of neuronal temporality, or the process of maintaining synaptic state between sequences of input, improves recall by over 50% at an operationally-relevant false alarm rate of 1 hr -1 .
Event-based cameras are cameras with high dynamic range that measure changes in light intensity at each pixel instead of capturing frames like traditional cameras. There are several event-based camera simulators for generating event-based datasets, each specialized towards a particular domain. However, none are designed specifically for robotic use-cases. This work addresses this issue using MuJoCo, a high performance physics engine; in combination with ESIM, an established event-generation method. To the authors' knowledge, this is the first robotic simulator tool for generating event-based datasets specifically designed for the robotics domain. Furthermore, to demonstrate its capabilities we generate an event-based visual dataset of industrial sockets, which is then used to train a SNN classifier.
Continual learning without catastrophic forgetting of previous experiences is an open general challenge for artificial neural networks, but is especially under-explored for artificial neural networks suitable to implement on neuromorphic platforms. An algorithmic understanding of how continual learning occurs in biological neural networks can inform solutions for artificial neural networks, especially in neuromorphic platforms whose biomimetic computing architectures lend themselves to more biofidelic algorithms. In this work, we derive an approach for generative-replay-based continual learning with a three-factor local learning rule based on recurrent connectivity in the insect's memory center, and characterize the model with a CIFAR-100 class incremental continual learning task. First, we investigate the properties of the model's internal representations and find that the high dimensional sparse representations enable this form of generative replay, and that these representations can be binary as required on spiking neuromorphic platforms with little detriment to model performance. Next, we derive a three-factor local learning rule by introducing simplifying assumptions to network updates from error backpropagation optimization which makes the learning rule biologically plausible (i.e., without weight transport) and amenable to neuromorphic implementation. Finally, we find that these simplifications enhance performance during gradient-based optimization for continual learning and when implemented locally achieve this increased performance. Overall, the outcomes of this work can inform a more detailed understanding for continual learning in this biological circuit, as well as introduce general approaches for neuromorphic continual learning.
Dynamic Vision Sensors (DVS) are fully asynchronous leading to excellent temporal resolution (≈1 μ s) and high throughput (>1 Giga-events per second) but are difficult to scale for higher resolutions (>1MegaPixels) due to the complex pixel design and higher static power consumption. Traditional CMOS imagers called Active Pixel Sensors (APS) employing synchronous frame scanning methods can scale to high resolutions (>100MegaPixels) but have severely limited temporal resolution due to the fixed frame rate, typically 120 frames per second (FPS). This work describes the study of a novel event-based camera implementing a Globally Asynchronous Locally Synchronous (GALS) architecture that can guarantee an equivalent frame rate of around 10000FPS for 1MP resolution and target throughput of 10Geps, breaking the barriers of conventional APS and DVS cameras. The architecture combines the APS frame scanning technique locally for low-resolution synchronous tiles, and asynchronous readout of the generated events, similar to DVS cameras, globally at the grid level (array of tiles). The study discusses the partitioning trade-off to implement the GALS architecture and determine the highest achievable event throughput against the frame sizes (resolution). Simulation results of different partitions of a 1MP resolution camera at peak event rate of 10Geps establishes the optimal partition as 32x32 asynchronous grid of tiles, with each tile consisting of 32x32 synchronous pixels. The analysis presented provides the impetus to design high-resolution event-based cameras using the proposed GALS architecture with optimal partitioning for extreme throughput applications.
Recent work in neuromorphic computing has proposed a range of new architectures for Spiking Neural Network (SNN)-based systems. However, neuromorphic design lacks a framework to facilitate exploration of different SNN-based architectures and aid with early design decisions. While there are various SNN simulators, none can be used to rapidly estimate latency and energy of different spiking architectures. We show that while current spiking designs differ in implementation, they have common features which can be represented as a generic architecture template. We describe an initial version of a framework that simulates a range of neuromorphic architectures at an abstract time-step granularity. We demonstrate our simulator by modeling Intel's Loihi platform, estimating time-varying energy and latency with less than 10% mean error for various sizes of a two-layer SNN.
