Structure Learning and the Parietal Cortex

We propose a theory of structure learning in the primate brain. We argue that the parietal cortex is critical for learning about relations among the objects and categories that populate a visual scene. We suggest that current deep learning models exhibit poor global scene understanding because they fail to perform the relational inferences that occur in the primate dorsal stream. We review studies of neural coding in primate posterior parietal cortex (PPC), drawing the conclusion that this brain area represents potentially high-dimensional inputs on a low-dimensional manifold that encodes relative the position of objects or features in physical space, and relations among entities an abstract conceptual space. We argue that this low-dimension code supports generalisation of relational information, even in nonspatial domains. Finally, we propose that structure learning is grounded in the actions that primates take when they reach for objects or fixate them with their eyes. We sketch a model of how this might occur in neural circuits.