Interpretable Vision Transformers in Image Classification Via SVDA

Vision Transformers (ViTs) have achieved state-of-the-art performance in image classification, yet their attention mechanisms often remain opaque and exhibit dense, non-structured behaviors. In this work, we adapt our previously proposed SVD-Inspired Attention (SVDA) mechanism to the ViT architecture, introducing a geometrically grounded formulation that enhances interpretability, sparsity, and spectral structure. We apply the use of interpretability indicators—originally proposed with SVDA—to monitor attention dynamics during training and assess structural properties of the learned representations. Experimental evaluations on four widely used benchmarks—CIFAR-10, FashionMNIST, CIFAR-100, and ImageNet-100—together with an additional pretrained fine-tuning study in a standard ViT setting show that SVDA preserves competitive classification behavior in our experimental settings while providing descriptive diagnostics of attention structure. In the pretrained setting, we integrate the exact SVDA operator into the late transformer blocks of a standard pretrained ViT and fine-tune on ImageNet-100, providing additional evidence that the proposed mechanism remains viable beyond compact from-scratch training. While the current framework offers descriptive insights rather than prescriptive guidance, our results establish SVDA as a comprehensive and informative tool for analyzing and developing structured attention models in computer vision. This work lays the foundation for future advances in explainable AI, spectral diagnostics, and attention-based model design.