Modern large-language-model infrastructure routes nearly every request through dense GPU compute regardless of reasoning depth required. A spiking neural network workload manager on a BrainChip Akida cluster under IBM Spectrum Symphony’s compute ontology routes across three tiers, dispatches in 5 ms and reduces GPU cost 30 to 50 percent. Beyond routing, the neuromorphic tier answers the same vLLM endpoints a GPU answers, holds 8,832 concurrent inference contexts across a 46-node fleet at 99.4 percent of ideal scaling and hot-swaps a hundred resident models at 11 ms each, which places the composition results of Grindrod, van der Made and Motta and Nanni on an operational substrate.

Real-time ISR demands sub-millisecond multimodal classification under tight SWaP, SLA and provenance constraints. A 46-node BrainChip Akida fleet under IBM Spectrum Symphony over GPFS fuses LPI radar, photonic RF, neuromorphic vision and OSINT modalities, delivering sub200 ms autonomous kill-chain decisions and a 30-watt 120-chip helmet-distributed perceptual layer. A demonstration program of sixty-seven builds exercises the arrangement across kinetic ISR, small-unit perception, orbital sensing and civilian remote sensing, and the composition results of Grindrod, van der Made and Motta and Nanni supply the dynamical, anatomical and empirical warrant for federating modest neuromorphic units rather than enlarging a single one.

We introduce VelocitySNN-Fuzzy AI, a novel neuromorphic system for real-time determination of three-dimensional velocity vectors (Vx, Vy, Vz) during Martian drone flight. Unlike Earth-Based UAVs, Mars operations cannot rely on GPS or conventional visual odometry due to the lack of infrastructure and the extreme lighting and terrain conditions. Our approach integrates a bio-inspired event-based camera, a Spiking Neural Network (SNN) developed using SpikingJelly, and an explainable Fuzzy AI Logic layer to produce reliable and interpretable velocity determination in challenging Mars-Like environments. Validated using publicly available Mars-Style drone datasets, the architecture demonstrates low-latency, high-dynamic-range perception suitable for small, power-constrained UAV platforms. VelocitySNN-Fuzzy AI determines 3D velocity for Martian drones using event-based vision, spiking neural networks, and fuzzy logic. Designed for GPS-denied, low-light environments, it enables real-time, low-power navigation. Validated on FPV drone datasets, it supports autonomous flight with interpretable outputs and deployment on neuromorphic hardware like Loihi and Akida.

Spiking neural network (SNN) hardware has gained significant interest due to its ability to process complex data in size, weight, and power (SWaP) constrained environments. Memristors, in particular, offer the potential to enhance SNN algorithms by providing analog domain acceleration with exceptional energy and throughput efficiency. Among the current SNN architectures, the Liquid State Machine (LSM), a form of Reservoir Computing (RC), stands out due to its low resource utilization and straightforward training process. In this paper, we present a custom memristor-based LSM circuit design with an online learning methodology. The proposed circuit implementing the LSM is designed using SPICE to ensure precise device level accuracy. Furthermore, we explore liquid connectivity tuning to facilitate a real-time and efficient design process. To assess the performance of our system, we evaluate it on multiple datasets, including MNIST, TI-46 spoken digits, acoustic drone recordings, and musical MIDI files. Our results demonstrate comparable accuracy while achieving significant power and energy savings when compared to existing LSM accelerators. Moreover, our design exhibits resilience in the presence of noise and neuron misfires. These findings highlight the potential of a memristor based LSM architecture to rival purely CMOS-based LSM implementations, offering robust and energy-efficient neuromorphic computing capabilities with memristive SNNs.

The event camera is researched, developed, and designed to imitate the human eye; it is a groundbreaking vision sensor with the following advantages over a standard camera: a net rate that is much faster, a latency that is far less, a high dynamic range, and it uses far less power. These fundamental properties assist in enabling the design of third-generation algorithms in Spiking Neural Networks intended to mimic the human brain and vision processing. Moreover, these fundamental properties enable swift robotics despite the challenges of motion blur and high latency that standard cameras face. Also, these properties should enable motion estimation from the surface features on Mars, which is difficult to achieve with standard cameras. For this research, the team is using a NASA Space use-case application. For this NASA Space application, the team has chosen a challenging motion-estimation task involving a Mars-based above-ground Helicopter beyond “Ingenuity” and the planet and surface of Mars. Event-based cameras have been gaining interest within the computer vision community. They are particularly suitable for applications with challenging temporal constraints and safety requirements. Thus, Event-based sensors are an excellent match for Spiking Neural Networks (SNNs), as coupling an asynchronous sensor with neuromorphic hardware can result in real-time systems with minimal power requirements. Moreover, methods to verify and validate event-based sensing platforms for space applications are lacking. In this paper, we investigate the addition of fuzzy logic models to arrive at an explainable SNN algorithm for the NASA space use-case. Ultimately, the team aims to develop a unified model yielding reasonably accurate optical flow estimates.

As artificial intelligence becomes increasingly popular, so too does the frequency it is attacked. One area where artificial intelligence models are being embraced heavily is avionics, where it is used in a variety of areas including intrusion detection, surveillance, radar, navigation, and more. These networks are vulnerable to a number of attacks, one of which being the adversarial attack. Adversarial attacks aim to perturb the input to the image classification model, which can lead to output misclassifications, lowering the overall model accuracy. This paper will demonstrate how such an attack can lower the accuracy of a convolutional neural network (CNN) trained on an aircraft image dataset. The perturbed inputs will be generated by a generative adversarial network (GAN) trained on the same dataset as the CNN. GANs consist of two neural networks, a generator and a discriminator. The generator will continually create new images based off of the dataset, while the discriminator decides whether or not the image is generated or from the reference dataset. This forms a feedback loop, where the generator continues to improve based on the output of the discriminator. Once trained, the outputs of the GAN will be used to misclassify the trained CNN in an attempt to lower the model’s accuracy. Following this, adversarial defenses such as feature squeezing and random smoothing will be discussed. These defenses attempt to mitigate the damage done by an attack, usually by adapting the model, modifying the input, or modifying the data the model is trained on. This can cause the accuracy of the model to slightly fall from its original accuracy, but with the benefit of being more resilient to adversarial attacks. We analyze this tradeoff and highlight the advantages of adversarial training in learning-enabled avionics systems.

The Neuromorphic based system contained in this research learns from the human pilot. The typical system for this type of research would have been designed for GPUs utilizing a Deep Neural Network. However, previous research indicates that GPUs require relatively large amounts of energy to operate and produce comparatively high amounts of heat. Moreover, due to Low Size Weight and Power (SWaP) constraints for Aircraft (A/C) and for Uncrewed Aerial Systems (UAS), GPUs may have to be off-board and or cloud computing utilized. Which may not be practical in situations. Therefore as an alternative the neuromorphic processor Loihi 2 was chosen. In this research, it was found that Loihi 2 solves these problems while showing to provide exceptional and robust performance. The Lava Sigma-Delta Neural Network (SDNN) inspired by PilotNet can be used in such a way to fly an aircraft or drone by learning from the human pilot using the neuromorphic Loihi 2. This study utilized actual live flight test data the researchers collected from live drone test flights to train an SDNN. During this research, a model was developed using the human hippocampus and visual cortex as they relate to the SDNN. The results of this study should enable an expanded autonomy envelope for A/C and UAS. This research will allow the creation of more scientific and engineering discovery and autonomy applications, ultimately empowering more mission efficacy by way of fast, on the edge, and stand alone computing. Demonstrated by the SDNN was a robust algorithm that emulated biological systems and generalized very well. To the best of our knowledge, this is the first in the world’s work that enables a UAS to be autonomously flown with Loihi 2, utilizing the SDNN.

The ultra-wideband, frequency agile and high duty cycle features of Low Probability of Intercept (LPI) radar pose serious concerns for passive radar detection systems that must efficiently identify LPI radar modulation schemes with very low latency and high accuracy. For these tasks, conventional deep learning techniques have often been employed, though this comes at the cost of increased model complexity and high energy consumption when implemented in tactically relevant scenarios with significant channel deterioration and size, weight, and power (SWaP) constrained platforms. In this paper, we propose a neuromorphic approach based on energy-efficient and robust Spiking Neural Networks (SNNs). To demonstrate the effectiveness of our proposed approach, we perform a challenging LPI radar waveform recognition task with a Spiking Convolutional Neural Network (SCNN). Simulation results show that the SCNN achieves a 13-waveform recognition accuracy of 99.1% at OdB SNR with 1.51µJ of total energy per inference. It is found that our system outperforms non-spiking models with a reduction in model size and three orders of magnitude lower energy consumption. To the best of our knowledge, this work presents the first spike-based implementation of an LPI radar waveform recognition system. Moreover, this paper highlights the potential for extremely low SWaP neuromorphic hardware deployment to enable intelligent electronic support systems that demand high throughput efficiency and ultra-low power.

Unmanned aerial vehicles (UAVs), also known as drones, are expected to play an integral role in next-generation wireless avionics networks. To help secure these networks from malicious activity, machine learning (ML) based approaches have been proposed to identify drones and their flight modes via direct radio frequency signals. These approaches frequently rely on cloudcentric methods for training, which present serious concerns, such as privacy leakage, resource burden, and undesirable latency. In response to these concerns, a distributed ML paradigm known as federated learning (FL) has been proposed that enables multiple drones to collaboratively train ML models by only exchanging model parameters and not the raw data itself. Unfortunately, the conventional FL framework is strongly dependent on a centralized aggregation server with multiple resource-constrained edge devices, making it susceptible to poisoning attacks and poor network utilization. Recently, it has been found that blockchain technology (BT) holds significant promise for securing and storing data in edge applications with high levels of trust. However, the expanding chain of security blocks consumes a significant amount of computing power, thus limiting its scalability. To bridge the gap between computational efficiency and security, we propose a blockchain-empowered and energy-efficient FL framework with neuromorphic edge devices. To assess the effectiveness of our proposed framework we perform a drone identification and flight mode detection task with a spiking neural network (SNN). Finally, we compare our neuromorphic approach with competitive alternatives to validate the energy and performance gains of our system.

Boeing "produced an airplane that had an unsafe condition" and admitted full responsibility for the second of its 737 MAX plane crash in Ethiopia in October 2018, killing 189 people. A major cause was found to be new flight control software, based on the AI (Artificial Intelligence) system called Maneuvering Characteristics Augmentation System (MCAS) which needed to be modified. It appears that the MCAS was originally designed as a limited AI system aimed at nose correction based on wind resistance and force sensors. However, in 2012, Boeing expanded that the MCAS and these changes, coupled with inadequate pilot training made it more vulnerable to malfunctioning. These unfortunate developments have further underscored the urgency to address airworthiness given the rise in autonomous systems and their integration into the aerospace domain. While there is an increased interest in effectively managing/regulating the AI certification problem for isolated complex functions, there is little work done on the certification of AI for aircraft that jointly evaluates System of Systems (SoS) integration in tandem with the human machine team dynamics involved in the engineering and deployment of AI-based systems. As a result, the development of airworthiness certification criteria will lag the development of state-of-the-art intelligent systems. This paper presents various considerations to enable the development of certification criteria for AI when consulting aerospace system design guidance and standards, including MIL-HDBK-516C, DO-178, and FAA's Airworthiness Certification Regulations. To allow for the validation and verification of AI-based systems to be safe, a systems engineering model is proposed to facilitate certification that includes an AI certification profile, verifiable implementation measures and a continuous process flow to ensure airworthiness that cannot be verified through traditional, test-based means. Furthermore, safety assurance approaches are identified for safety-critical systems that may include learning-enabled functions, with examples for both a Safety Critical Function Thread Analysis (SCFTA) and Failure Modes and Effects Testing (FMET). This paper represents a critical step towards the tailoring of airworthiness standards to include AI.

Spiking neural networks represent a transition from deep networks and tensor engines to more dynamic systems better suited for carrying out decisions based on temporal patterns within data. Furthermore, memristor based hardware for synaptic computation provides the potential for dramatic gains in terms of portability, power reduction, and throughput efficiency. In this paper we combine these ideas and present a memristor crossbar based implementation of a liquid state machine based on spiking neurons. The design is verified by demonstrating successful classification of MIDI (Musical Instrument Digital Interface) signals. Given the temporal nature of musical works, this dataset is well suited to show the effectiveness of the proposed hardware. The proposed circuit implementing this LSM is designed using SPICE to ensure accuracy at the device level, which aids in detailed analysis and circuit optimizations. Liquid layer activity is converted to state vectors using custom hardware that does not bottleneck throughput. Along with the proposed architecture, a neuron modeling procedure is described to match the software and hardware spiking neuron outputs, proving the feasibility of a full-scale implementation of the LSM system. Furthermore, various non-idealities are analyzed in the network, including misfiring neurons and noise associated with updating memristor conductance. We find that the proposed architecture is resilient to noise and fault-tolerant. In simulation, the performance of the memristor-based LSM is demonstrated on a musical genre classification experiment, where a recognition accuracy of 90% is achieved, compared to 91% accuracy in software. To the best of our knowledge this work represents the first end to end circuit implementation of an analog spiking neural network based reservoir computing system where memristors are responsible for all computation, with custom CMOS hardware for sampling the liquid layer. Finally, we tie the work together with an energy and timing analysis of the proposed system.

In the previous chapter, our rover needed the ability to extract information from images and recognize objects by their patterns and features. This chapter will outline a novel framework of autonomous aviation with the application of artificial intelligence in the form of a reinforcement learning agent. This agent learns flying skills by observing a pilot’s psychological reaction and flight path in a flight simulator. The framework consists of a gaming module that works as a flight simulator, a computer vision system that detects the pilot’s gestures, a flight dynamics analyzer for verifying the safety limits of the state space variables during a simulated flight, and a module to calculate the Q-function and the learned policy.

As the sophistication of unmanned aircraft systems (UAS) increases, so too does the strain they place on their operators. This strain is compounded when an operator is responsible for the control of multiple UAS working simultaneously, such as in a swarm. If an operator experiences too much mental strain, they may become overwhelmed and make costly errors which have catastrophic consequences. Current methods of swarm command and control are thoroughly researched though fail to address the impact on an operator(s) situation awareness. These architectures add complexity to an operation, while still relying solely on a human operator for command-and-control decision making on a macroscopic level. With the introduction of advanced neuromorphic computing hardware and 3 rd generation Artificial Intelligence systems, the strain placed on UAS operators could be delegated onto a hierarchical, autonomous UAS management entity. It would be responsible for interpreting commands given by the UAS operator, communicating between all assets in a swarm, delegating tasks to UAS within the swarm, making decisions based on the information available to the control system, and traveling with the swarm acting as a mobile pilot. This allows for an operator to take a more supervisory position, increasing their overall situation awareness, thus lowering mental workload. Neuromorphic technology allows our solution to be faster, lower power, scalable, generalized, and more autonomous when compared to architectures currently utilized.

In this chapter, our intrepid rover will explore the Egyptian catacombs. Any autonomous rover needs to know where it is in its environment; i.e., its current position. That way, the rover can travel from one location to the next without user input. To do this, the rover must construct an internal map of its environment as it travels. This chapter introduces the simultaneous localization and mapping (SLAM) algorithm. SLAM generates maps from navigation data, such as the odometry data from the rover’s wheel encoders and the laser range data from the LiDAR sensor.

Unmanned aerial vehicles (UAVs) need more autonomy. In light of inherent size, weight and power (SWaP) constraints, avionics with artificial intelligence implemented using neuromorphic technology offers a potential solution. We demonstrate intelligent drone control using spiking neural networks (SNNs), which can run on neuromorphic hardware. We present "BatSLAM", a modular SNN for autonomous localization, navigation, and control of UAVs. The bio-inspired algorithms are implemented using the neural modeling and simulation software Nengo, and are able to control the drone to autonomously perform a complex "house search" task in an AirSim simulated environment using only local sensor feedback. In this task, the drone is randomly placed and given a target object. The BatSLAM network localizes the drone, retrieves the target object location from memory, and then guides the drone along the most efficient path through the house to the target, avoiding obstacles and maneuvering through doors and stairways. We present benchmark results showing the BatSLAM network achieves 97.2% success rate in navigating to target objects in the house. To the best of our knowledge, the BatSLAM network presented here is the first in the world to carry out localization, navigation, and control in a fully spiking implementation.

Over the past few decades, the use of drones for applications such as defense, reconnaissance, agriculture, law enforcement, and others has dramatically increased. Although drones are useful for these applications, they can also be utilized to perform malicious activities, thus compromising the safety and integrity of physical infrastructures. To address this issue, various techniques have been developed to detect and identify drones, including radar, visual analysis, and radio-frequency signal processing. Furthermore, deep learning algorithms based on the recognition of acoustic drone features have been proposed to automate the detection process and overcome the current limitations in modern drone detection systems. This paper presents an auditory drone detection and identification system based on the sparse, event-driven communication nature of Spiking Neural Networks (SNNs). We investigate the use of a spiking reservoir computing model, known as a Liquid State Machine (LSM), that offers a computationally light alternative to the deep learning approaches of previous works. The LSM based auditory drone detection and identification system is demonstrated on a publicly available acoustic drone dataset, achieving an accuracy of 97.13% and 93.25% on the detection and identification tasks, respectively. To the best of our knowledge, this work presents the first spike-based implementation of an auditory drone recognition system. Moreover, this paper highlights the potential for low size, weight, and power neuromorphic hardware deployment for drone applications that may be limited to energy-constrained environments.