Representing complex structured data often involves a tradeoff between computational cost and representational quality, largely determined by the underlying encoding framework. Hyperdimensional Computing (HDC), also known as Vector Symbolic Architecture (VSA), has been recognized for its ability to represent complex structures through high-dimensional encoding. Conventional representation techniques in HDC, which in many cases rely on the cognitive operation of bundling for data memorization, suffer from quality and capacity limitations. Association through the binding operation offers a potential alternative, but its use is complicated by the difficult decomposition process, where classical factorization methods, such as resonator networks, can become challenging as the number of candidate factors and bound components increases. In this work, we explore a novel approach based on a quantum factorization algorithm inspired by Grover’s quantum search algorithm within HDC. This approach not only addresses the complexity challenges inherent in the decomposition of bound HDC representations, but also significantly mitigates noise and improves representational fidelity. Focusing on graph data structures, our framework Quark is analyzed through theoretical derivations and classical simulations of the quantum decoding process, demonstrating improved graph reconstruction quality across different graph sizes and dimensionalities. This integration of quantum computing and HDC provides a promising direction for scalable and high-fidelity graph representation.

Text-to-image diffusion models generate images through an iterative denoising process, so internal neural layers produce trajectories of activations rather than single static representations. Sparse autoencoders (SAEs) have recently been used to decompose diffusion activations into interpretable feature directions, but most approaches analyze activations at individual timesteps or condition on time rather than learning directly from full activation trajectories. In this work, we introduce residualized temporal SAEs for diffusion activation trajectories. We collect activations across denoising time, fit linear predictors between neighboring timesteps, and represent each trajectory using an initial activation together with residual components not explained by these linear dynamics. Training an SAE on this residualized representation encourages sparse latents to capture structure beyond what is linearly predictable. The residualized decoder directions can be mapped back into activation space, allowing each latent to be analyzed as a feature trajectory over denoising time. Through reconstruction and ablation studies, spatiotemporal feature analysis, and qualitative steering experiments on Stable Diffusion 1.5, we show that residualized temporal SAEs provide a useful framework for studying temporally structured diffusion activations.

Knowledge graphs play a crucial role in addressing the complexities of cybersecurity, as the increasing frequency and sophistication of cyber threats pose significant challenges to traditional defense technologies. In this paper, we propose a novel reasoning model, called INCYSER, that is tailored for cybersecurity. By leveraging Hyperdimensional Computing (HDC) as a symbolic and transparent computational model, INCYSER offers efficient and interpretable reasoning capabilities, ensuring reliable and trustworthy outcomes. Our model combines embedding-based unsupervised learning and HDC-based graph representation learning to construct a general representation for cybersecurity knowledge graphs, enabling diverse tasks including reasoning and general graph operations. Experimental evaluations demonstrate the effectiveness and efficiency of INCYSER, surpassing state-of-the-art models in link prediction and triple classification tasks. Additionally, a comprehensive ablation study examines the impact of various hyperparameters, showcasing the versatility of INCYSER. This work contributes to advancing the field of cybersecurity by introducing an interpretable and representation-based reasoning model for cybersecurity knowledge graphs.

Recent hardware accelerators for graph learning have largely overlooked knowledge graph reasoning (KGR), which demands more complex models and longer training times than typical graph tasks. Existing approaches rely on single or distributed GPUs to accelerate translational embedding models, but these general-purpose solutions lag in handling reinforcement learning-based KGR. To address this gap, we introduce KGAccel, the first domain-specific accelerator for RL-based KGR on FPGA. We develop a knowledge-graph compression method and propose a resource-aware mechanism that enables high-speed training even on smaller FPGAs. KGAccel achieves up to 65x speedup over CPU, 8x over GPU, and over 30x higher energy efficiency.

Graph analysis has emerged as a crucial field, offering versatile solutions for real-world data representation, from social networks to biological systems. However, the intricate nature of graphs often necessitates a degree of processing, such as learning mappings to a vector space, to perform analysis tasks like node classification and link prediction. A promising approach to this is Hyperdimensional Computing (HDC), inspired by neuroscience and mathematics. HDC utilizes high-dimensional vectors to efficiently manipulate complex data structures and perform operations like superposition and association, enhancing knowledge graph representations with contextual and semantic information. Nevertheless, addressing limitations in existing HDC-based approaches to graph representation is essential. This paper thoroughly explores these methods and presents ConfiGR: Configurable Graph Representation, a novel framework that introduces an adjustable design, enhancing its versatility across various graph types and tasks, ultimately boosting performance in multiple graph-related tasks.

Knowledge Graphs (KGs) have become a pivotal knowledge representation tool in machine learning, not only providing access to existing knowledge but also enabling the discovery of new knowledge through advanced applications. Among the scalable reasoning methods used for such applications, distributed graph embedding approaches, particularly GNNs, have become popular for large-scale graph-related tasks. However, many of these methods have limitations in their interpretability and fail to take into account structural similarity in their representation. Hyperdimensional Computing (HDC), also known as Vector Symbolic Architecture (VSA), addresses this issue by using well-defined cognitive operations on distributed representations of symbolic concepts. This work proposes and evaluates a new vector symbolic graph representation, CLOG, that preserves approximate structural similarity beyond edge correspondence and fundamentally differs from previous methods. The model's effectiveness in graph representation is evaluated through theoretical analysis, graph reconstruction experiments, and link prediction task, highlighting its efficiency and accuracy. This approach significantly advances the field by enhancing the capabilities of HDC in graph representation, representing a notable improvement over existing methods.

In recent times, a plethora of hardware accelerators has emerged, catering to graph learning applications. However, the focus has primarily been on accelerating graph analysis, graph clustering, and graph mining, with a lack of attention to knowledge graph reasoning. Graph reasoning requires a more complex model to handle the complicated knowledge graph compared to other graph learning tasks. A primary knowledge graph reasoning task is to find the implicit relations between entities of a given knowledge graph, which demands a significantly longer training time than traditional graph learning algorithms due to the model complexity. Therefore, it is essential to develop an acceleration method to mitigate the training cost for the practical deployment of this task. Prior work in this field has solely considered using a single GPU or distributed GPU cluster to accelerate translational embedding models. However, as demonstrated in this paper, such general-purpose GPUs don't provide satisfactory results for more complex reinforcement learning-based models. Hence, it becomes necessary to design customized domain-specific accelerators. This work proposes GraFlex, the first domain specific accelerator for reinforcement learning-based knowledge graph reasoning, implemented on FPGA. We first develop a compression method for knowledge graphs. Then, we explore FPGAs of different sizes, analyze their on-chip resources, and suggest a mechanism to achieve high-speed training on devices with insufficient resources using the aforementioned compression method.

In recent times, a plethora of hardware accelerators have been put forth for graph learning applications such as vertex classification and graph classification. However, previous works have paid little attention to Knowledge Graph Completion (KGC), a task that is well-known for its significantly higher algorithm complexity. The state-of-the-art KGC solutions based on graph convolution neural network (GCN) involve extensive vertex/relation embedding updates and complicated score functions, which are inherently cumbersome for acceleration. As a result, existing accelerator designs are no longer optimal, and a novel algorithm-hardware co-design for KG reasoning is needed. Recently, brain-inspired HyperDimensional Computing (HDC) has been introduced as a promising solution for lightweight machine learning, particularly for graph learning applications. In this paper, we leverage HDC for an intrinsically more efficient and acceleration-friendly KGC algorithm. We also co-design an acceleration framework named HDReason targeting FPGA platforms. On the algorithm level, HDReason achieves a balance between high reasoning accuracy, strong model interpretability, and less computation complexity. In terms of architecture, HDReason offers reconfigurability, high training throughput, and low energy consumption. When compared with NVIDIA RTX 4090 GPU, the proposed accelerator achieves an average 10.6x speedup and 65x energy efficiency improvement. When conducting cross-models and cross-platforms comparison, HDReason yields an average 4.2x higher performance and 3.4x better energy efficiency with similar accuracy versus the state-of-the-art FPGA-based GCN training platform.

IntroductionBrain-inspired computing has become an emerging field, where a growing number of works focus on developing algorithms that bring machine learning closer to human brains at the functional level. As one of the promising directions, Hyperdimensional Computing (HDC) is centered around the idea of having holographic and high-dimensional representation as the neural activities in our brains. Such representation is the fundamental enabler for the efficiency and robustness of HDC. However, existing HDC-based algorithms suffer from limitations within the encoder. To some extent, they all rely on manually selected encoders, meaning that the resulting representation is never adapted to the tasks at hand.MethodsIn this paper, we propose FLASH, a novel hyperdimensional learning method that incorporates an adaptive and learnable encoder design, aiming at better overall learning performance while maintaining good properties of HDC representation. Current HDC encoders leverage Random Fourier Features (RFF) for kernel correspondence and enable locality-preserving encoding. We propose to learn the encoder matrix distribution via gradient descent and effectively adapt the kernel for a more suitable HDC encoding.ResultsOur experiments on various regression datasets show that tuning the HDC encoder can significantly boost the accuracy, surpassing the current HDC-based algorithm and providing faster inference than other baselines, including RFF-based kernel ridge regression.DiscussionThe results indicate the importance of an adaptive encoder and customized high-dimensional representation in HDC.

The latest hardware accelerators proposed for graph applications primarily focus on graph neural networks (GNNs) and graph mining. High-level graph reasoning tasks, such as graph memorization and neighborhood reconstruction, have barely been addressed. Compared to low-level learning applications like node classification and clustering, high-level reasoning typically requires a more complex model to mimic human brain functionalities. Brain-inspired Hyper-Dimensional Computing (HDC) has recently introduced a promising lightweight and efficient machine learning solution, particularly for symbolic representation. General-purpose computing platforms (CPU/GPU) have been revealed to be inefficient for HDC applications. Therefore, it becomes essential to design a domain-specific accelerator targeting HDC-based graph reasoning algorithms. In this work, we propose the first domain-specific accelerator for HDC-based graph reasoning, HyperGRAF. We first develop a scheduler to balance the sparse matrix computation workloads, before parallelizing the hypervector calculations on two levels for the graph memorization task. Finally, we design a pipelinestyle matrix multiplication accelerator for the neighborhood reconstruction task. We evaluate our design under a wide range of generated graphs with different sizes and sparsity. The results show that HyperGRAF achieves over 100× improvement in both speedup and energy efficiency of graph reasoning compared to NVIDIA Jetson Orin.

Memorization is an essential functionality that enables today's machine learning algorithms to provide a high quality of learning and reasoning for each prediction. Memorization gives algorithms prior knowledge to keep the context and define confidence for their decision. Unfortunately, the existing deep learning algorithms have a weak and nontransparent notion of memorization. Brain-inspired HyperDimensional Computing (HDC) is introduced as a model of human memory. Therefore, it mimics several important functionalities of the brain memory by operating with a vector that is computationally tractable and mathematically rigorous in describing human cognition. In this manuscript, we introduce a brain-inspired system that represents HDC memorization capability over a graph of relations. We propose GrapHD, hyperdimensional memorization that represents graph-based information in high-dimensional space. GrapHD defines an encoding method representing complex graph structure while supporting both weighted and unweighted graphs. Our encoder spreads the information of all nodes and edges across into a full holistic representation so that no component is more responsible for storing any piece of information than another. Then, GrapHD defines several important cognitive functionalities over the encoded memory graph. These operations include memory reconstruction, information retrieval, graph matching, and shortest path. Our extensive evaluation shows that GrapHD: (1) significantly enhances learning capability by giving the notion of short/long term memorization to learning algorithms, (2) enables cognitive computing and reasoning over memorization graph, and (3) enables holographic brain-like computation with substantial robustness to noise and failure.

Recently, brain-inspired computing models have shown great potential to outperform today’s deep learning solutions in terms of robustness and energy efficiency. Particularly, Spiking Neural Networks (SNNs) and HyperDimensional Computing (HDC) have shown promising results in enabling efficient and robust cognitive learning. Despite the success, these two brain-inspired models have different strengths. While SNN mimics the physical properties of the human brain, HDC models the brain on a more abstract and functional level. Their design philosophies demonstrate complementary patterns that motivate their combination. With the help of the classical psychological model on memory, we propose SpikeHD, the first framework that fundamentally combines Spiking neural network and hyperdimensional computing. SpikeHD generates a scalable and strong cognitive learning system that better mimics brain functionality. SpikeHD exploits spiking neural networks to extract low-level features by preserving the spatial and temporal correlation of raw event-based spike data. Then, it utilizes HDC to operate over SNN output by mapping the signal into high-dimensional space, learning the abstract information, and classifying the data. Our extensive evaluation on a set of benchmark classification problems shows that SpikeHD provides the following benefit compared to SNN architecture: (1) significantly enhance learning capability by exploiting two-stage information processing, (2) enables substantial robustness to noise and failure, and (3) reduces the network size and required parameters to learn complex information.

Face detection is an essential component of many tasks in computer vision with several applications. However, existing deep learning solutions are significantly slow and inefficient to enable face detection on embedded platforms. In this paper, we propose HDFace, a novel framework for highly efficient and robust face detection. HDFace exploits HyperDimensional Computing (HDC) as a neurally-inspired computational paradigm that mimics important brain functionalities towards high-efficiency and noise-tolerant computation. We first develop a novel technique that enables HDC to perform stochastic arithmetic computations over binary hypervectors. Next, we expand these arithmetic for efficient and robust processing of feature extraction algorithms in hyperspace. Finally, we develop an adaptive hyperdimensional classification algorithm for effective and robust face detection. We evaluate the effectiveness of HDFace on large-scale emotion detection and face detection applications. Our results indicate that HDFace provides, on average, 6.1X (4.6X) speedup and 3.0X (12.1X) energy efficiency as compared to neural networks running on CPU (FPGA), respectively.