Categories of Neural Networks

We introduce new categories of neural networks that we hope will help us expose and explore their underlying structure and relationships. Our categories represent neural networks as objects and present structure for comparing neural networks with the same number of layers. In this paper we discuss the construction, properties, and currently known limitations – all with specific, illustrative examples. There are a number of directions this work can move towards, and we conclude with a section on future work summarizing several of these directions.