Integrating a Gaussian Classifier into a CNN

The assignment of natural images to categories is an important problem both in machine learning and in the study of human cognition. Visual categorization models in psychology examine strategies of human categorization, which computer vision models seek to effectively discriminate between categories. Convolutional Neural Networks (CNNs) are the current state-of-the-art in computer vision — the training performance of these discriminative models is approaching near perfect levels, but their generalizability leaves room for improvement. On the other hand, cognitive models are density estimators: generative not discriminative. In particular, prototypical models categorize by measuring distance from the ‘mean’ or ‘ideal’ prototype of a category. Given the related nature of these fields, it is interesting to ask whether the research results from one can inform and enrich the goals of the other. Motivated by the notions of testing prototype theory on representations created by CNNs, and exploring ways for the CNN to make more human-like representations of categories, this project integrates a differentiable Gaussian classifier in a Convolutional Neural Network. In particular, it models classes in the CIFAR-10 dataset as multivariate Gaussian distributions, and creates three custom Keras classification layers corresponding to nested constraints on the covariance of the distributions. This project finds that these class representation do not impede accuracy of predictions, and significantly decreases the discrepancy between human and machine-learned distributions of judgements.