Adaptive Variability in Humans, Pigeons, and Rats.

Adaptively variable behavior can be advantageous in various fields such as sports (unpredictability), art (creativity), science (innovation), and problem-solving (thinking outside the box). Although previous studies identified experimental conditions under which humans and non-human animals show increased variable decision making, we have only a limited understanding of its underlying cognitive mechanisms. Using a reinforcement learning model, we simulate three different theorized strategies in an adversarial reward learning environment that requires very high variability. These mechanisms are (1) relying on a stochastic generator, (2) increasing one’s learning rate, or (3) upvaluing unchosen actions. A systematic parameter search and a policy gradient meta-learning algorithm both show that agents can adaptively increase variability with each of those strategies. Next, we fitted our model on existing datasets with humans, pigeons, and rats as subjects in adversarial environments. While all three species can engage in highly variable behavior, we find that only humans upvalue unchosen actions as a strategy to achieve variability.