Parametrization of Sparse Distributed Representations for Vector Data Classification

The growing demand for Artificial Intelligence (AI) capabilities in diverse application settings is having a proportional impact on computational processing and memory required for building and deploying such models. This has triggered a transition from large AI models to small models and energy-efficient machine learning techniques. Vector Symbolic Architectures (VSA) provide an effective framework for this transition to energy efficient learning by encoding and manipulating data in high-dimensional vector space. Particularly, sparse distributed representations in VSA significantly improve computational and memory efficiency of most machine learning algorithms. In this paper, we examine the parametrization of sparse distributed representations in VSA for vector data classification tasks. Focusing on dimensionality and density, the paper explores strategies for encoding vector data using Sparse Block Codes (SBC), Sparse Binary Distributed Representations (SBDR), and dense Holographic Reduced Representation (HRR). We use a compositional approach to encode vector data and systematically evaluate parameter choices to optimize classification performance. Through a detailed parametrization using 121 UCI classification datasets, we identify key design considerations for sparse distributed representations. These experiments demonstrate that the accuracy improves rapidly with increasing dimensionality followed by saturation. For SBC, moderate structured density is consistently high performing: 25% density yields the highest average test accuracy. For SBDR, moderate-density CDT settings have the highest performance, while extreme densities degrade performance. Under optimal settings, sparse representations match state-of-the-art dense performance: SBC achieves 80.98%, SBDR 80.13%, and dense HRR 81.02% mean accuracy over the 121 datasets, providing practical guidance on achieving performance comparable to state-of-the-art VSA methods, highlighting their effectiveness for efficient and scalable machine learning.