The Basal Ganglia As a Policy-Based Controller

Successful action selection and execution requires fine-grained control across a continuous and complex range of movement parameters and contexts. Behavioral decisions are rarely simple binaries. While indexing and selecting between individual discrete actions has been an attractive model in reductionist conditions, the myriad state-action values an animal must keep track of make biological bookkeeping untenable. Here, we lay out the key problems a brain must solve when choosing and optimizing how to interact with our diverse world. We then review several behavioral and physiological studies that suggest a reappraisal of how the brain, and specifically the basal ganglia, solves these problems.