Spiking neural network (SNN) hardware has gained significant interest due to its ability to process complex data in size, weight, and power (SWaP) constrained environments. Memristors, in particular, offer the potential to enhance SNN algorithms by providing analog domain acceleration with exceptional energy and throughput efficiency. Among the current SNN architectures, the Liquid State Machine (LSM), a form of Reservoir Computing (RC), stands out due to its low resource utilization and straightforward training process. In this paper, we present a custom memristor-based LSM circuit design with an online learning methodology. The proposed circuit implementing the LSM is designed using SPICE to ensure precise device level accuracy. Furthermore, we explore liquid connectivity tuning to facilitate a real-time and efficient design process. To assess the performance of our system, we evaluate it on multiple datasets, including MNIST, TI-46 spoken digits, acoustic drone recordings, and musical MIDI files. Our results demonstrate comparable accuracy while achieving significant power and energy savings when compared to existing LSM accelerators. Moreover, our design exhibits resilience in the presence of noise and neuron misfires. These findings highlight the potential of a memristor based LSM architecture to rival purely CMOS-based LSM implementations, offering robust and energy-efficient neuromorphic computing capabilities with memristive SNNs.

The event camera is researched, developed, and designed to imitate the human eye; it is a groundbreaking vision sensor with the following advantages over a standard camera: a net rate that is much faster, a latency that is far less, a high dynamic range, and it uses far less power. These fundamental properties assist in enabling the design of third-generation algorithms in Spiking Neural Networks intended to mimic the human brain and vision processing. Moreover, these fundamental properties enable swift robotics despite the challenges of motion blur and high latency that standard cameras face. Also, these properties should enable motion estimation from the surface features on Mars, which is difficult to achieve with standard cameras. For this research, the team is using a NASA Space use-case application. For this NASA Space application, the team has chosen a challenging motion-estimation task involving a Mars-based above-ground Helicopter beyond “Ingenuity” and the planet and surface of Mars. Event-based cameras have been gaining interest within the computer vision community. They are particularly suitable for applications with challenging temporal constraints and safety requirements. Thus, Event-based sensors are an excellent match for Spiking Neural Networks (SNNs), as coupling an asynchronous sensor with neuromorphic hardware can result in real-time systems with minimal power requirements. Moreover, methods to verify and validate event-based sensing platforms for space applications are lacking. In this paper, we investigate the addition of fuzzy logic models to arrive at an explainable SNN algorithm for the NASA space use-case. Ultimately, the team aims to develop a unified model yielding reasonably accurate optical flow estimates.

As artificial intelligence becomes increasingly popular, so too does the frequency it is attacked. One area where artificial intelligence models are being embraced heavily is avionics, where it is used in a variety of areas including intrusion detection, surveillance, radar, navigation, and more. These networks are vulnerable to a number of attacks, one of which being the adversarial attack. Adversarial attacks aim to perturb the input to the image classification model, which can lead to output misclassifications, lowering the overall model accuracy. This paper will demonstrate how such an attack can lower the accuracy of a convolutional neural network (CNN) trained on an aircraft image dataset. The perturbed inputs will be generated by a generative adversarial network (GAN) trained on the same dataset as the CNN. GANs consist of two neural networks, a generator and a discriminator. The generator will continually create new images based off of the dataset, while the discriminator decides whether or not the image is generated or from the reference dataset. This forms a feedback loop, where the generator continues to improve based on the output of the discriminator. Once trained, the outputs of the GAN will be used to misclassify the trained CNN in an attempt to lower the model’s accuracy. Following this, adversarial defenses such as feature squeezing and random smoothing will be discussed. These defenses attempt to mitigate the damage done by an attack, usually by adapting the model, modifying the input, or modifying the data the model is trained on. This can cause the accuracy of the model to slightly fall from its original accuracy, but with the benefit of being more resilient to adversarial attacks. We analyze this tradeoff and highlight the advantages of adversarial training in learning-enabled avionics systems.

The Neuromorphic based system contained in this research learns from the human pilot. The typical system for this type of research would have been designed for GPUs utilizing a Deep Neural Network. However, previous research indicates that GPUs require relatively large amounts of energy to operate and produce comparatively high amounts of heat. Moreover, due to Low Size Weight and Power (SWaP) constraints for Aircraft (A/C) and for Uncrewed Aerial Systems (UAS), GPUs may have to be off-board and or cloud computing utilized. Which may not be practical in situations. Therefore as an alternative the neuromorphic processor Loihi 2 was chosen. In this research, it was found that Loihi 2 solves these problems while showing to provide exceptional and robust performance. The Lava Sigma-Delta Neural Network (SDNN) inspired by PilotNet can be used in such a way to fly an aircraft or drone by learning from the human pilot using the neuromorphic Loihi 2. This study utilized actual live flight test data the researchers collected from live drone test flights to train an SDNN. During this research, a model was developed using the human hippocampus and visual cortex as they relate to the SDNN. The results of this study should enable an expanded autonomy envelope for A/C and UAS. This research will allow the creation of more scientific and engineering discovery and autonomy applications, ultimately empowering more mission efficacy by way of fast, on the edge, and stand alone computing. Demonstrated by the SDNN was a robust algorithm that emulated biological systems and generalized very well. To the best of our knowledge, this is the first in the world’s work that enables a UAS to be autonomously flown with Loihi 2, utilizing the SDNN.

The ultra-wideband, frequency agile and high duty cycle features of Low Probability of Intercept (LPI) radar pose serious concerns for passive radar detection systems that must efficiently identify LPI radar modulation schemes with very low latency and high accuracy. For these tasks, conventional deep learning techniques have often been employed, though this comes at the cost of increased model complexity and high energy consumption when implemented in tactically relevant scenarios with significant channel deterioration and size, weight, and power (SWaP) constrained platforms. In this paper, we propose a neuromorphic approach based on energy-efficient and robust Spiking Neural Networks (SNNs). To demonstrate the effectiveness of our proposed approach, we perform a challenging LPI radar waveform recognition task with a Spiking Convolutional Neural Network (SCNN). Simulation results show that the SCNN achieves a 13-waveform recognition accuracy of 99.1% at OdB SNR with 1.51µJ of total energy per inference. It is found that our system outperforms non-spiking models with a reduction in model size and three orders of magnitude lower energy consumption. To the best of our knowledge, this work presents the first spike-based implementation of an LPI radar waveform recognition system. Moreover, this paper highlights the potential for extremely low SWaP neuromorphic hardware deployment to enable intelligent electronic support systems that demand high throughput efficiency and ultra-low power.

Unmanned aerial vehicles (UAVs), also known as drones, are expected to play an integral role in next-generation wireless avionics networks. To help secure these networks from malicious activity, machine learning (ML) based approaches have been proposed to identify drones and their flight modes via direct radio frequency signals. These approaches frequently rely on cloudcentric methods for training, which present serious concerns, such as privacy leakage, resource burden, and undesirable latency. In response to these concerns, a distributed ML paradigm known as federated learning (FL) has been proposed that enables multiple drones to collaboratively train ML models by only exchanging model parameters and not the raw data itself. Unfortunately, the conventional FL framework is strongly dependent on a centralized aggregation server with multiple resource-constrained edge devices, making it susceptible to poisoning attacks and poor network utilization. Recently, it has been found that blockchain technology (BT) holds significant promise for securing and storing data in edge applications with high levels of trust. However, the expanding chain of security blocks consumes a significant amount of computing power, thus limiting its scalability. To bridge the gap between computational efficiency and security, we propose a blockchain-empowered and energy-efficient FL framework with neuromorphic edge devices. To assess the effectiveness of our proposed framework we perform a drone identification and flight mode detection task with a spiking neural network (SNN). Finally, we compare our neuromorphic approach with competitive alternatives to validate the energy and performance gains of our system.

Boeing "produced an airplane that had an unsafe condition" and admitted full responsibility for the second of its 737 MAX plane crash in Ethiopia in October 2018, killing 189 people. A major cause was found to be new flight control software, based on the AI (Artificial Intelligence) system called Maneuvering Characteristics Augmentation System (MCAS) which needed to be modified. It appears that the MCAS was originally designed as a limited AI system aimed at nose correction based on wind resistance and force sensors. However, in 2012, Boeing expanded that the MCAS and these changes, coupled with inadequate pilot training made it more vulnerable to malfunctioning. These unfortunate developments have further underscored the urgency to address airworthiness given the rise in autonomous systems and their integration into the aerospace domain. While there is an increased interest in effectively managing/regulating the AI certification problem for isolated complex functions, there is little work done on the certification of AI for aircraft that jointly evaluates System of Systems (SoS) integration in tandem with the human machine team dynamics involved in the engineering and deployment of AI-based systems. As a result, the development of airworthiness certification criteria will lag the development of state-of-the-art intelligent systems. This paper presents various considerations to enable the development of certification criteria for AI when consulting aerospace system design guidance and standards, including MIL-HDBK-516C, DO-178, and FAA's Airworthiness Certification Regulations. To allow for the validation and verification of AI-based systems to be safe, a systems engineering model is proposed to facilitate certification that includes an AI certification profile, verifiable implementation measures and a continuous process flow to ensure airworthiness that cannot be verified through traditional, test-based means. Furthermore, safety assurance approaches are identified for safety-critical systems that may include learning-enabled functions, with examples for both a Safety Critical Function Thread Analysis (SCFTA) and Failure Modes and Effects Testing (FMET). This paper represents a critical step towards the tailoring of airworthiness standards to include AI.

Spiking neural networks represent a transition from deep networks and tensor engines to more dynamic systems better suited for carrying out decisions based on temporal patterns within data. Furthermore, memristor based hardware for synaptic computation provides the potential for dramatic gains in terms of portability, power reduction, and throughput efficiency. In this paper we combine these ideas and present a memristor crossbar based implementation of a liquid state machine based on spiking neurons. The design is verified by demonstrating successful classification of MIDI (Musical Instrument Digital Interface) signals. Given the temporal nature of musical works, this dataset is well suited to show the effectiveness of the proposed hardware. The proposed circuit implementing this LSM is designed using SPICE to ensure accuracy at the device level, which aids in detailed analysis and circuit optimizations. Liquid layer activity is converted to state vectors using custom hardware that does not bottleneck throughput. Along with the proposed architecture, a neuron modeling procedure is described to match the software and hardware spiking neuron outputs, proving the feasibility of a full-scale implementation of the LSM system. Furthermore, various non-idealities are analyzed in the network, including misfiring neurons and noise associated with updating memristor conductance. We find that the proposed architecture is resilient to noise and fault-tolerant. In simulation, the performance of the memristor-based LSM is demonstrated on a musical genre classification experiment, where a recognition accuracy of 90% is achieved, compared to 91% accuracy in software. To the best of our knowledge this work represents the first end to end circuit implementation of an analog spiking neural network based reservoir computing system where memristors are responsible for all computation, with custom CMOS hardware for sampling the liquid layer. Finally, we tie the work together with an energy and timing analysis of the proposed system.

Over the past few decades, the use of drones for applications such as defense, reconnaissance, agriculture, law enforcement, and others has dramatically increased. Although drones are useful for these applications, they can also be utilized to perform malicious activities, thus compromising the safety and integrity of physical infrastructures. To address this issue, various techniques have been developed to detect and identify drones, including radar, visual analysis, and radio-frequency signal processing. Furthermore, deep learning algorithms based on the recognition of acoustic drone features have been proposed to automate the detection process and overcome the current limitations in modern drone detection systems. This paper presents an auditory drone detection and identification system based on the sparse, event-driven communication nature of Spiking Neural Networks (SNNs). We investigate the use of a spiking reservoir computing model, known as a Liquid State Machine (LSM), that offers a computationally light alternative to the deep learning approaches of previous works. The LSM based auditory drone detection and identification system is demonstrated on a publicly available acoustic drone dataset, achieving an accuracy of 97.13% and 93.25% on the detection and identification tasks, respectively. To the best of our knowledge, this work presents the first spike-based implementation of an auditory drone recognition system. Moreover, this paper highlights the potential for low size, weight, and power neuromorphic hardware deployment for drone applications that may be limited to energy-constrained environments.

An often overlooked but equally important aspect of unmanned aerial system (UAS) design is the security of their networking protocols and how they deal with cyberattacks. In this context, cyberattacks are malicious attempts to monitor or modify incoming and outgoing data from the system. These attacks could target anywhere in the system where a transfer of data occurs but are most common in the transfer of data between the control station and the UAS. A compromise in the networking system of a UAS could result in a variety of issues including increased network latency between the control station and the UAS, temporary loss of control over the UAS, or a complete loss of the UAS. A complete loss of the system could result in the UAS being disabled, crashing, or the attacker overtaking command and control of the platform, all of which would be done with little to no alert to the operator. Fortunately, the majority of higher-end, enterprise, and government UAS platforms are aware of these threats and take actions to mitigate them. However, as the consumer market continues to grow and prices continue to drop, network security may be overlooked or ignored in favor of producing the lowest cost product possible. Additionally, these commercial off-the-shelf UAS often use uniform, standardized frequency bands, autopilots, and security measures, meaning a cyberattack could be developed to affect a wide variety of models with minimal changes. This paper will focus on a low-cost educational-use UAS and test its resilience to a variety of cyberattack methods, including man-in-the-middle attacks, spoofing of data, and distributed denial-of-service attacks. Following this experiment will be a discussion of current cybersecurity practices for counteracting these attacks and how they can be applied onboard a UAS. Although in this case the cyberattacks were tested against a simpler platform, the methods discussed are applicable to any UAS platform attempting to defend against such cyberattack methods.

Spiking neural network hardware offers a high performance, power-efficient and robust platform for the processing of complex data. Many of these systems require supervised learning, which poses a challenge when using gradient-based algorithms due to the discontinuous properties of SNNs. Memristor based hardware can offer gains in portability, power reduction, and throughput efficiency when compared to pure CMOS. This paper proposes a memristor-based spiking liquid state machine (LSM). The inherent dynamics of the LSM permit the use of supervised learning without backpropagation for weight updates. To carry out the design space evaluation of the LSM for optimal hardware performance, several temporal signal classification tasks are performed. It is found that the binary neuron activations in the output layer improve testing accuracy by 3.7% and 5% for classification, while reducing training time. A power and energy analysis of the proposed hardware is presented, resulting in an approximately 50% reduction in power consumption and cycle energy.