Deep belief networks and cortical algorithms: A comparative study for supervised classification

The failure of shallow neural network architectures in replicating human intelligence led the machine learning community to focus on deep learning, to computationally match human intelligence. The wide availability of increasing computing power coupled with the development of more efficient training algorithms have allowed the implementation of deep learning principles in a manner and span that had not been previously possible. This has led to the inception of deep architectures that capitalize on recent advances in artificial intelligence and insights from cognitive neuroscience to provide better learning solutions. In this paper, we discuss two such algorithms that represent different approaches to deep learning with varied levels of maturity. The more mature but less biologically inspired Deep Belief Network (DBN) and the more biologically grounded Cortical Algorithms (CA) are first introduced to give readers a bird’s eye view of the higher-level concepts that make up these algorithms, as well as some of their technical underpinnings and applications. Their theoretical computational complexity is then derived before comparing their empirical performance on some publicly available classification datasets. Multiple network architectures were compared and showed that CA outperformed DBN on most datasets, with the best network architecture consisting of six hidden layers.