Spiking neural networks (SNNs) are neuromorphic systems that emulate certain aspects of biological neural tissue, offering potential advantages in energy efficiency and speed by for example leveraging sparsity. While CMOS-based electronic SNN hardware has shown promise, scalability and parallelism challenges remain. Photonics provides a promising platform for SNNs due to the speed of excitable photonic devices standing in as neurons and the parallelism and low-latency of optical signal conduction. Here, we present a photonic SNN comprising 40 000 neurons using off-the-shelf components, including a spatial light modulator and a CMOS camera, enabling scalable and cost-effective implementations for photonic SNN proof of concept studies. The system is governed by a modified Ikeda map, where adding slow inhibitory feedback forcing introduces excitability akin to biological dynamics. Using latency encoding and sparsity, the network achieves 83.5% accuracy on MNIST handwritten digits using only 22% of neurons, and 77.5% with only 8.5% of neurons. Training is performed via liquid state machine concepts combined with the hardware-compatible simultaneous perturbation stochastic approximation algorithm, marking its first use in photonic neural networks. This demonstration integrates photonic nonlinearity, excitability, and sparse computation, paving the way for efficient large-scale photonic neuromorphic systems.

For Edge AI applications, deploying online learning and adaptation on resource-constrained embedded devices can deal with low-latency sensor-generated data streams in changing environments. However, since maintaining low-latency and power-efficient inference is paramount at the Edge, online learning and adaptation on the device should impose minimal additional overhead for inference. With this goal in mind, we explore energy-efficient learning and adaptation on-device for streaming-data Edge AI applications using Spiking Neural Networks (SNNs), which follow the principles of brain-inspired computing, such as high-parallelism, neuron co-located memory and compute, and event-driven processing. We propose EON-1, a brain-inspired processor for near-sensor extreme-edge online feature extraction that integrates a fast online learning and adaptation algorithm. We report results of only 1% energy overhead for learning, by far the lowest overhead when compared to other SoTA solutions, while attaining comparable inference accuracy. Furthermore, we demonstrate that EON-1 is up for the challenge of low-latency processing of HD and UHD streaming video in real-time, with learning enabled.

Classification and recognition tasks performed on photonic hardware-based neural networks often require at least one offline computational step, such as in the increasingly popular reservoir computing paradigm. Removing this offline step can significantly improve the response time and energy efficiency of such systems. We present numerical simulations of different algorithms that utilize ultrafast photonic spiking neurons as receptive fields to allow for image recognition without an offline computing step. In particular, we discuss the merits of event, spike-time and rank-order based algorithms adapted to this system. These techniques have the potential to significantly improve the efficiency and effectiveness of optical classification systems, minimizing the number of spiking nodes required for a given task and leveraging the parallelism offered by photonic hardware.

Modern computation based on von Neumann architecture is now a mature cutting-edge science. In the von Neumann architecture, processing and memory units are implemented as separate blocks interchanging data intensively and continuously. This data transfer is responsible for a large part of the power consumption. The next generation computer technology is expected to solve problems at the exascale with 1018 calculations each second. Even though these future computers will be incredibly powerful, if they are based on von Neumann type architectures, they will consume between 20 and 30 megawatts of power and will not have intrinsic physically built-in capabilities to learn or deal with complex data as our brain does. These needs can be addressed by neuromorphic computing systems which are inspired by the biological concepts of the human brain. This new generation of computers has the potential to be used for the storage and processing of large amounts of digital information with much lower power consumption than conventional processors. Among their potential future applications, an important niche is moving the control from data centers to edge devices. The aim of this roadmap is to present a snapshot of the present state of neuromorphic technology and provide an opinion on the challenges and opportunities that the future holds in the major areas of neuromorphic technology, namely materials, devices, neuromorphic circuits, neuromorphic algorithms, applications, and ethics. The roadmap is a collection of perspectives where leading researchers in the neuromorphic community provide their own view about the current state and the future challenges for each research area. We hope that this roadmap will be a useful resource by providing a concise yet comprehensive introduction to readers outside this field, for those who are just entering the field, as well as providing future perspectives for those who are well established in the neuromorphic computing community.

Spiking neural networks (SNNs) using time-to-first-spike (TTFS) codes, in which neurons fire at most once, are appealing for rapid and low power processing. In this theoretical paper, we focus on information coding and decoding in those networks, and introduce a new unifying mathematical framework that allows the comparison of various coding schemes. In an early proposal, called rank-order coding (ROC), neurons are maximally activated when inputs arrive in the order of their synaptic weights, thanks to a shunting inhibition mechanism that progressively desensitizes the neurons as spikes arrive. In another proposal, called NoM coding, only the first N spikes of M input neurons are propagated, and these “first spike patterns” can be readout by downstream neurons with homogeneous weights and no desensitization: as a result, the exact order between the first spikes does not matter. This paper also introduces a third option—“Ranked-NoM” (R-NoM), which combines features from both ROC and NoM coding schemes: only the first N input spikes are propagated, but their order is readout by downstream neurons thanks to inhomogeneous weights and linear desensitization. The unifying mathematical framework allows the three codes to be compared in terms of discriminability, which measures to what extent a neuron responds more strongly to its preferred input spike pattern than to random patterns. This discriminability turns out to be much higher for R-NoM than for the other codes, especially in the early phase of the responses. We also argue that R-NoM is much more hardware-friendly than the original ROC proposal, although NoM remains the easiest to implement in hardware because it only requires binary synapses.

What is the content and the format of visual memories in Long Term Memory (LTM)? Is it similar in adults and children? To address these issues, we investigated, in both adults and 9-year-old children, how visual LTM is affected over time and whether visual vs semantic features are affected differentially. In a learning phase, participants were exposed to hundreds of meaningless and meaningful images presented once or twice for either 120 ms or 1920 ms. Memory was assessed using a recognition task either immediately after learning or after a delay of three or six weeks. The results suggest that multiple and extended exposures are crucial for retaining an image for several weeks. Although a benefit was observed in the meaningful condition when memory was assessed immediately after learning, this benefit tended to disappear over weeks, especially when the images were presented twice for 1920 ms. This pattern was observed for both adults and children. Together, the results call into question the dominant models of LTM for images: although semantic information enhances the encoding & maintaining of images in LTM when assessed immediately, this seems not critical for LTM over weeks.

Dennis V. Christensen, Regina Dittmann, Bernabe Linares-Barranco, Abu Sebastian, Manuel Le Gallo, Andrea Redaelli, Stefan Slesazeck, Thomas Mikolajick, Sabina Spiga, Stephan Menzel, Ilia Valov, Gianluca Milano, Carlo Ricciardi, Shi-Jun Liang, Feng Miao, Mario Lanza, Tyler J. Quill, Scott T. Keene, Alberto Salleo, Julie Grollier, Danijela Markovic, Alice Mizrahi, Peng Yao, J. Joshua Yang, Giacomo Indiveri, John Paul Strachan, Suman Datta, Elisa Vianello, Alexandre Valentian, Johannes Feldmann, Xuan Li, Wolfram H.P. Pernice, Harish Bhaskaran, Emre Neftci, Srikanth Ramaswamy, Jonathan Tapson, Franz Scherr, Wolfgang Maass, Priyadarshini Panda, Youngeun Kim, Gouhei Tanaka, Simon Thorpe, Chiara Bartolozzi, Thomas A. Cleland, Christoph Posch, Shih-Chii Liu, Arnab Neelim Mazumder, Morteza Hosseini, Tinoosh Mohsenin, Elisa Donati, Silvia Tolu, Roberto Galeazzi, Martin Ejsing Christensen, Sune Holm, Daniele Ielmini, N. Pryds

BACKGROUND:Recordings with tetrodes have proven to be more effective in isolating single neuron spiking activity than with single microwires. However, tetrodes have never been used in humans. We report on the characteristics, safety, compatibility with clinical intracranial recordings in epileptic patients, and performance, of a new type of hybrid electrode equipped with tetrodes. NEW METHOD:240 standard clinical macroelectrodes and 102 hybrid electrodes were implanted in 28 patients. Hybrids (diameter 800 μm) are made of 6 or 9 macro-contacts and 2 or 3 tetrodes (diameter 70-80 μm). RESULTS:No clinical complication or adverse event was associated with the hybrids. Impedance and noise of recordings were stable over time. The design enabled multiscale spatial analyses that revealed physiopathological events which were sometimes specific to one tetrode, but could not be recorded on the macro-contacts. After spike sorting, the single-unit yield was similar to other hybrid electrodes and was sometimes as high as >10 neurons per tetrode. COMPARISON WITH EXISTING METHOD(S):This new hybrid electrode has a smaller diameter than other available hybrid electrodes. It provides novel spatial information due to the configuration of the tetrodes. The single-unit yield appears promising. CONCLUSIONS:This new hybrid electrode is safe, easy to use, and works satisfactorily for conducting multi-scale seizure and physiological analyses.

The commonly accepted “simple-to-complex” model of visual processing in the brain posits that visual tasks on complex objects such as faces are based on representations in high-level visual areas. Yet, recent experimental data showing the visual system’s ability to localize faces in natural images within 100ms ([Crouzet et al., 2010][1]) challenge the prevalent hierarchical description of the visual system, and instead suggest the hypothesis of face-selectivity in early visual areas. In the present study, we tested this hypothesis with human participants in two eye tracking experiments, an fMRI experiment and an EEG experiment. We found converging evidence for neural representations selective for upright faces in V1/V2, with latencies starting around 40 ms post-stimulus onset. Our findings suggest a revision of the standard “simple-to-complex” model of hierarchical visual processing.Significance statement Visual processing in the brain is classically described as a series of stages with increasingly complex object representations: early visual areas encode simple visual features (such as oriented bars), and high-level visual areas encode representations for complex objects (such as faces). In the present study, we provide behavioral, fMRI, and EEG evidence for representations of complex objects – namely faces – in early visual areas. Our results challenge the standard “simple-to-complex” model of visual processing, suggesting that it needs to be revised to include neural representations for faces at the lowest levels of the visual hierarchy. Such early object representations would permit the rapid and precise localization of complex objects, as has previously been reported for the object class of faces. [1]: #ref-7

There is currently a replication crisis in many fields of neuroscience and psychology, with some estimates claiming up to 64% of research in psychological science is not reproducible. Three common culprits which have been suspected to cause the failure to replicate such studies are small sample sizes, “hypothesizing after the results are known,” and “p-hacking.” Here, we introduce accurate stimulus timing as an additional possibility. Accurate stimulus onset timing is critical to almost all psychophysical research. Auditory, visual, or manual response time stimulus onsets are typically sent through wires to various machines that record data such as: eye gaze positions, electroencephalography, stereo electroencephalography, and electrocorticography. These stimulus onsets are collated and analyzed according to experimental condition. If there is variability in the temporal accuracy of the delivery of these onsets to external systems, the quality of the resulting data and scientific analyses will degrade. Here, we describe an approximately $200 Arduino based system and associated open-source codebase which achieved a 5.34 microsecond delay from the inputs to the outputs while electrically opto-isolating the connected external systems. Using an oscilloscope, the device is configurable for different environmental conditions particular to each laboratory (e.g. light sensor type, screen type, speaker type, stimulus type, temperature, etc). This low-cost open-source project delivered electrically isolated stimulus onset Transistor-Transistor Logic triggers with a median precision of 5.34 microseconds and was successfully tested with 7 different external systems that record eye and neurological data.

In the context of word learning, it is commonly assumed that repetition is required for young children to form and maintain in memory an association between a novel word and its corresponding object. For instance, at 2 years of age, children are able to dis-ambiguate word-related situations in one shot but are not able to further retain this newly acquired knowledge. It has been proposed that multiple fast-mapping experiences would be required to promote word retention or that the inferential reasoning needs to be accompanied by explicit labeling of the target. We hypothesized that when 2-year-olds simply encounter an unambiguous learning context, word learning may be fast and maintained in time. We also assumed that, under this condition, even a single exposure to an object would be sufficient to form a memory trace of its name that would survive a delay. To test these hypotheses, 2- and 4-year-olds were ostensively taught three arbitrary word-object pairs using a 15-s video sequence during which each object was manually displayed and labeled three times in a row. Retention was measured after a 30-min distractive period using a forced-choice procedure. Our results provide evidence that declarative memory does not need repetition to be formed and maintained, for at least a 30-min period, by children as young as 2 years. This finding suggests that the mechanisms required for extremely rapid and robust word acquisition not only are present in preschoolers with developed language and cognitive skills but also are already operative at a younger age. (C) 2020 The Authors. Published by Elsevier Inc.

Here, we provide an analysis of the microsaccades that occurred during continuous visual search and targeting of small faces that we pasted either into cluttered background photos or into a simple gray background. Subjects continuously used their eyes to target singular 3-degree upright or inverted faces in changing scenes. As soon as the participant's gaze reached the target face, a new face was displayed in a different and random location. Regardless of the experimental context (e.g. background scene, no background scene), or target eccentricity (from 4 to 20 degrees of visual angle), we found that the microsaccade rate dropped to near zero levels within only 12 milliseconds after stimulus onset. There were almost never any microsaccades after stimulus onset and before the first saccade to the face. One subject completed 118 consecutive trials without a single microsaccade. However, in about 20% of the trials, there was a single microsaccade that occurred almost immediately after the preceding saccade's offset. These microsaccades were task oriented because their facial landmark targeting distributions matched those of saccades within both the upright and inverted face conditions. Our findings show that a single feedforward pass through the visual hierarchy for each stimulus is likely all that is needed to effectuate prolonged continuous visual search. In addition, we provide evidence that microsaccades can serve perceptual functions like correcting saccades or effectuating task-oriented goals during continuous visual search.

Unlike familiarity, recollection involves the ability to reconstruct mentally previous events that results in a strong sense of reliving. According to the reinstatement hypothesis, this specific feature emerges from the reactivation of cortical patterns involved during information exposure. Over time, the retrieval of specific details becomes more difficult, and memories become increasingly supported by familiarity judgments. The multiple trace theory (MTT) explains the gradual loss of episodic details by a transformation in the memory representation, a view that is not shared by the standard consolidation model. In this study, we tested the MTT in light of the reinstatement hypothesis. The temporal dynamics of mental imagery from long-term memory were investigated and tracked over the passage of time. Participant EEG activity was recorded during the recall of short audiovisual clips that had been watched 3 weeks, 1 day, or a few hours beforehand. The recall of the audiovisual clips was assessed using a Remember/Know/New procedure, and snapshots of clips were used as recall cues. The decoding matrices obtained from the multivariate pattern analyses revealed sustained patterns that occurred at long latencies (>500 msec poststimulus onset) that faded away over the retention intervals and that emerged from the same neural processes. Overall, our data provide further evidence toward the MTT and give new insights into the exploration of our “mind's eye.”

Human observers readily detect targets and repetitions in streams of rapidly presented visual stimuli. It seems intuitive that regularly spaced repeating items should be easier to detect than irregularly spaced ones, since regularity adds predictability and in addition has ecological relevance. Here, we show that this is not necessarily the case, and we point out the intrinsic difficulty in addressing this question. We presented long RSVP streams of never-before-seen natural images containing repetition sequences; an image appearing six times interleaved by one or more non-repeating distractors, and asked participants to detect the repetitions and to afterwards identify the repeated images. We found that the ability to detect and memorize repeated images was preserved even with irregular sequences, and conclude that temporal regularity is not a key factor for detection and memory for repeating images in RSVP streams. These findings have implications for models of repetition processing.

Behavioral studies in humans indicate that peripheral vision can do object recognition to some extent. Moreover, recent studies have shown that some information from brain regions retinotopic to visual periphery is somehow fed back to regions retinotopic to the fovea and disrupting this feedback impairs object recognition in human. However, it is unclear to what extent the information in visual periphery contributes to human object categorization. Here, we designed two series of rapid object categorization tasks to first investigate the performance of human peripheral vision in categorizing natural object images at different eccentricities and abstraction levels (superordinate, basic, and subordinate). Then, using a delayed foveal noise mask, we studied how modulating the foveal representation impacts peripheral object categorization at any of the abstraction levels. We found that peripheral vision can quickly and accurately accomplish superordinate categorization, while its performance in finer categorization levels dramatically drops as the object presents further in the periphery. Also, we found that a 300-ms delayed foveal noise mask can significantly disturb categorization performance in basic and subordinate levels, while it has no effect on the superordinate level. Our results suggest that human peripheral vision can easily process objects at high abstraction levels, and the information is fed back to foveal vision to prime foveal cortex for finer categorizations when a saccade is made toward the target object.

Human observers readily detect targets in stimuli presented briefly and in rapid succession. Here, we show that even without predefined targets, humans can spot repetitions in streams of thousands of images. We presented sequences of natural images reoccurring a number of times interleaved with either one or two distractors, and we asked participants to detect the repetitions and to identify the repeated images after a delay that could last for minutes. Performance improved with the number of repeated-image presentations up to a ceiling around seven repetitions and was above chance even after only two to three presentations. The task was easiest for slow streams; performance dropped with increasing image-presentation rate but stabilized above 15 Hz and remained well above chance even at 120 Hz. To summarize, we reveal that the human brain has an impressive capacity to detect repetitions in rapid-serial-visual-presentation streams and to remember repeated images over a time course of minutes.

Objective: The mechanisms underlying epileptogenicity in tuberous sclerosis complex (TSC) are poorly understood. Methods: We analysed neuronal spiking activity (84 neurons), fast ripples (FRs), local field potentials and intracranial electroencephalogram during interictal epileptiform discharges (IEDs) in the tuber and perituber of a patient using novel hybrid electrodes equipped with tetrodes. Results: IEDs were recorded in the tuber and perituber. FRs were recorded only in the tuber and only with the microelectrodes. A larger proportion of neurons in the tuber (57%) than in the perituber (17%) had firing-rates modulated around IEDs. Conclusions: A multi-scale analysis of neuronal activity, FRs and IEDs indicates a gradient of epileptogenicity running from the tuber to the perituber. Significance: We demonstrate, for the first time in vivo, a gradient of epileptogenicity from the tuber to the perituber, which paves the way for future models of epilepsy in TSC. Our results also question the extent of the neurosurgical resection, including or not the perituber, that needs to be made in these patients. (C) 2019 International Federation of Clinical Neurophysiology. Published by Elsevier B.V.

•We used a bio-inspired deep convolutional spiking neural network with latency-coding.•We trained the low (resp. top) layers with STDP (resp. reward-modulated STDP).•Accuracy was 97.2% on MNIST, without requiring an external classifier.•Reward-modulated STDP favors diagnostic features, while STDP favors frequent ones.•The proposed neuron-based decision-making layer is suitable for energy-efficient hardware implementation.