This paper presents a spiking neural network (SNN) implementation which employs unsupervised feature extraction using spike timing dependent plasticity (STDP) to classify 8 different radioisotopes. With the implementation, the accuracy could reach 80% during training and overall testing accuracy of 72%. The whole network was implemented on SpiNNaker, a spiking neural network emulation platform. This work shows that unsupervised STDP, an SNN native training method, can be applied to the classification task of RIID to provide event-based training as well as inference.
Event driven cameras have the potential to revolutionise the real-time visual sensory processing paradigm. These asynchronous sensors detect change in the environment with low latency and high dynamic range, allowing for orders of magnitude faster systems than the state of the art using intensity cameras. On the other hand, deep artificial neural networks have refashioned machine vision in the last decade, greatly expanding the reach of viable tasks, supported by the creation of many large scale image datasets. In this work, we present a modality to leverage these large scale datasets for the purpose of training off-the-shelf deep learning architectures and re-appropriating them for event-based tasks. To this end, we describe an event representation, EROS, and a method to convert images to an EROS-like representation such that image datasets can train neural networks for event driven applications.
This paper is focused on the interpolation of a signal modeled by a random process from a set of discretetime measurements. The process of signal sampling is studied as a conditioning of a random process at instants of its discrete-time observations. The analysis shows that even if an input is modeled by a stationary Gaussian process, the conditional random process with a set of observations at sampling instants, is still Gaussian but non-stationary. By the adoption of standard properties of the multivariate normal distribution, we derive the mean of the conditional process, which is at the same time the minimum mean-square error (MMSE) predictor for signal reconstruction based on the given information represented by the observations. It is shown that for Gaussian signals, the MMSE predictor is a linear function of the observed data. For bandlimited signals, the conditional MMSE predictor coincides to the well-known MMSE reconstruction derived by Yen based on the deterministic approach. Although the approach covers any measurement scheme, possibly non-uniform in time, the study is narrowed down to the interpolation of the signal from its level-crossing samples. The accuracy of the MMSE reconstruction has been verified by simulations.
Event cameras are promising sensors for on-line and real-time vision tasks due to their high temporal resolution, low latency, and redundant static data elimination. Many vision algorithms use some form of spatial convolution (i.e. spatial pattern detection) as a fundamental component. However, additional consideration must be taken for event cameras, as the visual signal is asynchronous and sparse. While elegant methods have been proposed for event-based convolutions, they are unsuitable for real scenarios due to their inefficient processing pipeline and subsequent low event-throughput. This paper presents an efficient implementation based on decoupling the event-based computations from the computationally heavy convolutions, increasing the maximum event processing rate by 15. 92 × to over 10 million events/second, while still maintaining the event-based paradigm of asynchronous input and output. Results on public datasets with modern 640 × 480 event-camera recordings show that the proposed implementation achieves real-time processing with minimal impact on the convolution result, while the prior state-of-the-art results in a latency of over 1 second.
Nowadays, controlled systems with constant time sampling are widely spread. Conversely, the event-triggered paradigm is highly appealing since it promises to reduce computation and data flow when the dynamics of the system requires little or no control update, while allowing more intense refresh during transition phases. The objective here is to extend this opportunity and to design a fully event-based control architecture by considering not only the controller but the whole sensorimotor chain from perception to action, with the hope of even better gains in resource utilization. The targeted application is the visual servoing of a ball-on-beam system. The proposal includes a visual sensor, perception and control algorithms, as well as an actuator, all of which being event driven. Moreover, the proposed strategy is implemented on an embedded CPU-based platform. Beyond the innovative design, experimental results highlight savings in computational cost and bandwidth, with even slightly better performances.
Event-based image sensors are a new class of sensors developed thanks to non-uniform sampling and asynchronous technology, which overcomes many image sensor limitations such as a high throughput or a huge power consumption. As their behavior and outputs are really different from traditional image sensors, the produced data stream imposes to completely rethink image processing. Indeed, dedicated algorithms are mandatory to take advantage of this specific data stream, known as Address Event Representation (AER). This paper presents an improved and dedicated event-by-event clustering algorithm allowing the object detection in a noisy environment which is still performant with a SNR of 1/4. We measure high recall and precision for different simulated scenarios with multiple objects and show an improvement compared to the previous algorithm. The approach especially demonstrates a low computational complexity and a reduced memory footprint, which is perfectly suited for low-cost and low-power embedded image sensing applications.
For a robot to compete at the game of air-hockey requires the ability to track the fast-moving puck, and fast reaction of its control system. Event-cameras can be used to solve the visual tracking task in order to overcome problems of motion blur and/or high processing requirements that come from when using traditional RGB cameras. Each pixel of an event-camera responds independently to change in light, resulting in a high frequency, low-latency update of the puck position. A vision-in-the-loop robot control can then maintain stability with much faster movements. In this paper, we introduce the control loop for an iCub robot to follow the position of the puck with its head motion. We evaluate the accuracy and stability of the iCub motion as the latency of the tracked position is varied from 1 ms to 30 ms, achievable in real-time with the event-camera, eventually resulting in control failure. We conclude that the event-driven tracking paradigm is an enabling technology for unlocking smooth dynamic robot motions from vision, also for tasks beyond air-hockey.
Cell sorting and counting technology has been broadly adopted for medical diagnosis, cell-based therapy, and biological research. Microscopy operates with image capture that is subject to an extremely constrained field-of-view, and even slow-moving targets may undergo motion blur, ghosting, and other movement-induced artifacts, which will ultimately degrade performance in developing machine learning models to perform cell sorting, detection, and tracking. Frame-based sensors are especially susceptible to these issues, and it is highly costly to overcome them with modern but conventional CMOS sensing technologies. We provide an early demonstration of a proof-of-concept system, with the overarching goals of curating a neuromorphic imaging cytometry (NIC) dataset, multimodal analysis techniques, and associated deep-learning models. We are working towards this goal by utilising an event-based camera to perform flow-imaging cytometry to capture cells in motion and train neural networks capable of identifying their morphology (size and shape) and identities. We propose that implementing a neuromorphic sensory system or developing a new class of event-based cameras customised for this purpose with our sorting strategy will unbind the applications from the constraints of framerate and provide a cost-efficient, reproducible and high-throughput imaging mechanism. While we target this early work for cell sorting, this novel idea is the first stepping-stone towards a new type of high-throughput and automated high-content image analysis system and screening instrument.
This paper presents a high-speed, object detection algorithm that leverages data from an event-based camera and a spike-based, cellular neural network framework for morphological processing. Event-based data flows into the algorithm in a time-serial, asynchronous fashion, but the algorithm and subsequent architecture description lends itself towards a parallel approach. A cellular neural-network (CNN) is composed of multimodal processing elements that provide the means to spatiotemporally filter event data, but also are used to apply a cascade of piece-wise linear functions in a synchronous fashion. When applied in succession, these morphological operations form object “blobs”, produce shape skeletons, and place centroids. Over an event stream, this rapid centroid placement provides a means to perform low-latency object detection in an embedded framework. Using processing intervals of 25ms and assuming a clock of 100 MHz, a computational latency of around 5.5μs is incurred and an estimated 161uW is consumed by the morphological algorithm thus providing a promising solution for event-based, embedded processing.
Spiking Neural Networks are a recent and new neural network design approach that promises tremendous improvements in power efficiency, computation efficiency, and processing latency. They do so by using asynchronous spike-based data flow, event-based signal generation, processing, and modifying the neuron model to resemble biological neurons closely. While some initial works have shown significant initial evidence of applicability to common deep learning tasks, their applications in complex real-world tasks have been relatively low. In this work, we first illustrate the applicability of spiking neural networks to a complex deep learning task, namely LiDAR based 3D object detection for automated driving. Secondly, we make a step-by-step demonstration of simulating spiking behavior using a pre-trained Convolutional Neural Network. We closely model essential aspects of spiking neural networks in simulation and achieve equivalent run-time and accuracy on a GPU. We expect significant improvements in power efficiency when the model is implemented on neuromorphic hardware.
The third generation of artificial intelligence (AI) introduced by neuromorphic computing is revolutionizing the way robots and autonomous systems can sense the world, process the information, and interact with their environment. Research towards fulfilling the promises of high flexibility, energy efficiency, and robustness of neuromorphic systems is widely supported by software tools for simulating spiking neural networks, and hardware integration (neuromorphic processors). Yet, while efforts have been made on neuromorphic vision (event-based cameras), it is worth noting that most of the sensors available for robotics remain inherently incompatible with neuromorphic computing, where information is encoded into spikes. To facilitate the use of traditional sensors, we need to convert the output signals into streams of spikes, i.e., a series of events (+ 1,- 1) along with their corresponding timestamps. In this paper, we propose a review of the coding algorithms from a robotics perspective and further supported by a benchmark to assess their performance. We also introduce a ROS (Robot Operating System) toolbox to encode and decode input signals coming from any type of sensor available on a robot. This initiative is meant to stimulate and facilitate robotic integration of neuromorphic AI, with the opportunity to adapt traditional off-the-shelf sensors to spiking neural nets within one of the most powerful robotic tools, ROS.
The Power quality (PQ) disturbances causes rigorous issues in classical and smart grids, industries. The performance of power networks can be affected by these intermittent events. The identification of PQ disturbances and an effective prevention of such events are essential. In this framework, vital aspects are a precise understanding as a first step can be followed by a real-time treatment of the PQ disturbances in the future later on. The PQ signals are acquired by using the event-driven A/D converters (EDADCs). The acquired signal is segmented by using novel event-driven signal selection technique. Afterwards, the segmented signal pertinent features are extracted by using an effective adaptive rate time-domain analysis approach. These features are passed to the robust machine-learning based classifiers to realize an automated identification of the PQ disturbances. The system secures 13.26-fold compression gain compared to the conventional fix-rate counterparts. The highest classification precision of 99.44% is secured. It confirms that the suggested method can be integrated in contemporary automated PQ identifiers.
Bubble detection and correction logic is vital in modern data capture devices to solve bubbles in the output thermometer codes due to non-linearities in the scale causing negative bin widths. Previous bubble correction techniques are either unsuitable for short pulse widths and multiple registration (ones-encoder) or have a very short range (all other methods). In this paper, we propose a hardware technique to detect and correct bubbles up to the length of the pulse width while preserving position information using a hybrid between the ones-encoder and a single stage of a modified insertion sort. This design was shown to meet timing on a Xilinx Artix-7 FPGA at 100 MHz or above using only 13% of the device, demonstrating hardware-viability. The design is also fully-pipelined to demonstrate high bandwidths. The limitations of the algorithm are stated and some possible improvements are suggested.
Simultaneous perturbation stochastic approximation was previously shown to provide good results for optimizing resource consumption under dynamically changing workload. We employ this approach to develop dynamic voltage-frequency scaling (DVFS) governor for Android OS. We design experimental methodology and show our algorithm performance compared to common Android DVFS governors. A link to the source code is also provided.
Time-to-Digital Converters (TDCs) are important devices in many systems, such as Time-of-Flight, frequency locking, nuclear experiments and metrology, and quantum versions of all these. Therefore, research into TDCs is important to bring these technologies to wider audiences. There was plenty of TDC innovation in 2020 despite the global pandemic, and in this review paper we summarise the research related to TDC architectures. We show four review and analysis papers, eight papers on application-specific integrated circuits, five papers on field-programmable gate arrays, one paper on discrete TDCs, and four papers on novel applications of the TDC core. This trend of excellent research looks set to continue into 2021, with 21 papers already mentioning TDC despite major conferences such as NoMeTDC and I2MTC not having occurred as of writing.
Stochastic subgradient algorithms (SSAs) are widely studied owing to their applications in distributed and online learning. However, in a distributed setting, their sub-linear convergence rates tend to attract a large number of information exchanges that raise the overall communication burden. In order to reduce this burden, in this paper, we design two static stochastic event-based broadcasting protocols that operate in conjunction with SSAs to address a set-constrained distributed optimization problem (DOP). We address two notions of stochastic convergence, namely, almost sure and mean convergence; for each of these notions we design event-based broadcasting protocols, specifically, the stochastic event-thresholds. Subsequently, we illustrate the design via a numerical example and provide comparisons to evaluate its performance against the existing event-based protocols.
This paper presents a high precision Time-to-Digital Converter (TDC) implemented in a low-end, low power Field Programmable Gate Array (FPGA). It applies the interpolation method for fine time measurement, and a multi-chain tapped-delay line (TDL) structure is used to achieve high precision. To avoid bubble error and to decrease dead time, a Wallace-tree encoder with pipeline structure is utilized in each chain, for encoding the 234-bit thermometer code from the delay taps. A double-channel TDC with different numbers of chains is implemented in a Xilinx Artix-7 100T FPGA to verify the design concept and to study the performances in detail. Test results show that with eight chains implemented for each channel, the best precision of leading-edge measurement (channel to channel) can reach 6.4 ps in root mean square (RMS). The LUT resource occupancy of the double-channel TDC (together with auxiliary circuits) is about 9.77%, which shows good potential for expanding the channel numbers for high density, low cost applications.
The battery is a crucial element of modern power systems and it is utilized habitually in different vital applications such as electric vehicles, drones, avionics and mobile phones. Among various batteries technologies the Li-Ion batteries are widely used. It is mainly because of their compactness, longer life and high power capacity. On the other hand, due to the disadvantage of Li-ion batteries being expensive, their use is monitored using battery management systems (BMSs) to optimize their performance and ensure they last longer. The extensive processing resources that modern BMSs need can result in higher overhead power consumption. This study focuses on upgrading the present Li-ion BMSs through redesigning their associative data acquisition and processing chains differently. It aims at enhancing the data acquisition and estimation mechanisms for the Li-ion batteries' capacities. It utilizes a novel event-driven mechanism for extracting the intended Li-Ion cell parameters. The event-driven approach brings notable compression gain compared to fix-rate conventional counterparts. The mined attributes are onward conveyed to the robust machine learning algorithms for prediction. The 5-fold cross-validation approach is used for prediction performance evaluation. The achieved correlation coefficient and minimum Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) are respectively 0.9996, 0.0038 and 0.0054 respectively. It shows the feasibility of incorporating the proposed approach in contemporary BMSs.
The impulsive Goodwin's oscillator (IGO) is a hybrid model that captures complex dynamics arising in continuous systems controlled by pulse-modulated (event-based) feedback. Being conceived to describe pulsatile endocrine regulation, it has also found applications in e.g. pharmacokinetics. The original version of the IGO assumes the continuous part of the model to be a chain of first-order blocks. This paper explores the nonlinear phenomena arising due to the introduction of a local continuous feedback as suggested by the endocrine applications. The effects caused by a nonlinear feedback law parameterized by a Hill function are compared to those arising due to a simpler and previously treated case of affine feedback law. The hybrid dynamics of the IGO are reduced to a (discrete) Poincaré map governing the propagation of the model's continuous states through the firing instants of the impulsive feedback. Bifurcation analysis of the map reveals in particular that both the local Hill function and affine feedback can lead to multistability, which phenomenon has not been observed in the usual IGO model.
This paper presents a new approach for dead-time minimization while preserving low resource usage and high resolution in FPGA-based time-to-digital (TDC) converters. The proposed TDC architecture can be employed in applications in which many events need to be detected in a short time, such as time-of-flight positron emission tomography (ToF-PET) applications. The presented architecture consists of a toggling input stage, a tapped delay line (TDL), a dual-mode counter-based encoder, a coarse counter, and a bin width calibration stage. The minimum dead-time of TDL TDCs is two clock cycles. The proposed architecture reduced dead-time to one clock cycle. The measurement results of the proposed low-resources TDC in an Artix-7 FPGA show [-0.80, 1.34] LSB differential nonlinearity (DNL) and [-0.73, 1.97] LSB integral non-linearity (INL). The measured LSB size and single-shot precision (SSP) are 22.1 ps and 28.43 ps, respectively.
