Making Models Disagree to Learn How Brains Compute

Computational hypotheses about brain information processing can be expressed in neural network models. Neuroscientists have begun to compare such models in terms of their alignment with neural and behavioural data. The high parametric capacity of these models is essential to their ability to capture cognitive processes but also enables them to approximate arbitrary functions, making distinct models difficult to discriminate experimentally. Model comparisons using stimuli sampled from the training distribution often fail to reveal differences. This challenge can be met by optimizing stimulus sets for model discrimination and by leveraging out-of-distribution generalization as a severe test. This Review explains the emerging methods for optimizing stimuli to adjudicate among neural network models. These methods seek stimulus sets that are controversial among the models in that they make the models disagree in their predictions of the experimental data. We discuss the choices researchers must make, including a prior over candidate stimuli (such as naturalistic images), a measure of the power to discriminate among alternative models and a procedure for selecting or synthesizing stimulus sets that maximize model-comparison power. Historically, researchers have chosen either natural or artificial stimuli for a given study, prioritizing ecological validity or model-comparison power, respectively. Tempered by a prior, controversial stimuli offer a synthesis of these classical approaches, combining the greater ecological validity of naturalistic stimuli with the greater power for model comparison enabled by artificial stimuli. We offer a unified perspective on current work, drawing connections to Bayesian optimal experimental design.