Eye-Hand Coordination Develops from Active Multimodal Compression

During their first months of life, infants learn to coordinate their perceptions and actions across different modalities. For example, eye-hand coordination relies on combining visual and proprioceptive sensory inputs for controlling eye and hand movements. What drives the development and calibration of such coordination? Here, we put forward a multimodal hierarchical extension of the Active Efficient Coding framework to learn a simple form of eye-hand coordination. By learning to actively compress visual and proprioceptive inputs into a combined multimodal representation, our embodied infant model learns to make eye movements to track an object held in its hand. We find that the abstract multimodal representation improves the tracking accuracy, but only if it emerges after the establishment of the single-modality systems. This suggests the existence of a “less-is-more” effect for the development of coordinated multimodal sensorimotor behaviors.