Flexible, goal-directed behavior depends on the ability to update value representations in response to changing contingencies. This ability depends on distributed brain networks, calling for use of whole-brain imaging. Widely used in human research, whole-brain imaging in rodents has been a major challenge that has only recently been properly addressed. Using functional MRI in behaving mice performing a go/no-go odor discrimination task, we compared neural activity during initial cue-reward learning (acquisition) and subsequent contingency reversal. To link neural activity to underlying learning processes, we modeled value updating using a model-free reinforcement-learning algorithm. Trial-by-trial estimates of state-action values allowed us to dissociate acquisition from reversal-related signals, revealing that ventral striatal responses tracked expected value during acquisition, whereas reversal learning additionally recruited the periaqueductal gray (PAG), a midbrain structure classically linked to threat processing and aversive learning. PAG activity closely followed model-derived signatures of reversal learning, implicating it in the suppression of previously rewarded actions and in updating behavior in the absence of explicit punishment. These findings reveal a previously unrecognized computational role for the PAG in value-based decision-making and cognitive flexibility, and substantiate task-fMRI as a powerful tool to study the rodent brain at a mesoscale resolution.

Our experiences contain countless details that may be important in the future, yet we rarely know which will matter and which will not. This uncertainty poses a difficult challenge for adaptive decision-making, as failing to preserve relevant information can prevent us from making good choices later on. One solution to this challenge is to store detailed memories of individual experiences that can be flexibly accessed whenever their details become relevant. By allowing us to store and recall specific events in vivid detail, the human episodic memory system provides exactly this capacity. Yet, whether and how this ability supports adaptive behaviour is poorly understood. Here we aimed to determine whether people use detailed episodic memories to make decisions when future task demands are uncertain. We hypothesized that the episodic memory system’s ability to store events in great detail allows us to reference any of these details if they later become relevant. We tested this hypothesis using a novel decision-making task in which participants encoded individual events with multiple features and later made decisions based on these features to maximize their earnings. Across 5 experiments (total n = 535), we found that participants referenced episodic memories during decisions in feature-rich environments and that they did so specifically when it was unclear at encoding which features would be needed in the future. Overall, these findings reveal a fundamental adaptive function of episodic memory, showing how its rich representational capacity enables flexible decision-making under uncertainty. Nicholas and Mattar found that people use episodic memory to make decisions when it is unclear what will be needed in the future. These findings reveal how the rich representational capacity of episodic memory enables flexible decision-making.

Patients with Parkinson's disease (PD) are impaired at incremental reward-based learning. It is typically assumed that this impairment reflects a loss of striatal dopamine. However, many open questions remain about the nature of reward-based learning deficits in PD. Recent studies have found that even simple reward-based learning tasks rely on a combination of cognitive and computational strategies, including one-shot episodic memory. These findings raise questions about how incremental learning and episodic memory contribute to decision-making in PD. We tested healthy participants (n = 26; 14 males and 12 females) and patients with PD (n = 26; 16 males and 10 females), both on- and off-dopamine replacement medication, on a task designed to differentiate between the contributions of incremental learning and episodic memory to reward-based learning and decision-making. We found that PD patients performed equally well as healthy controls when using episodic memory but were impaired at incremental rewardbased learning. Dopamine replacement medication remediated this deficit and enhanced subsequent episodic memory for the value of motivationally relevant stimuli. These results demonstrate that while PD patients are impaired at learning about reward from trial-and-error, their ability to encode memories for the value of one-shot experiences is intact.

We are often faced with decisions we have never encountered before, requiring us to infer possible outcomes before making a choice. Computational theories suggest that one way to make these types of decisions is by accessing and linking related experiences stored in memory. Past work has shown that such memory-based preference construction can occur at a number of different timepoints relative to the moment a decision is made. Some studies have found that memories are integrated at the time a decision is faced (reactively) while others found that memory integration happens earlier, when memories were initially encoded (proactively). Here we offer a resolution to this inconsistency, demonstrating that these two strategies tradeoff rationally as a function of the associative structure of memory. We use fMRI to decode patterns of brain responses unique to categories of images in memory and find that proactive memory access is more common and allows more efficient inference. However, we also find that participants use reactive access when choice options are linked to a larger number of memory associations. Together, these results indicate that the brain judiciously conducts proactive inference by accessing memories ahead of time when conditions make this strategy more favorable.

Patients with Parkinson's disease are impaired at incremental reward-based learning. It is typically assumed that this impairment reflects a loss of striatal dopamine. However, many open questions remain about the nature of reward-based learning deficits in Parkinson's. Recent studies have found that a combination of different cognitive and computational strategies contribute even to simple reward-based learning tasks, suggesting a possible role for episodic memory. These findings raise critical questions about how incremental learning and episodic memory interact to support learning from past experience and what their relative contributions are to impaired decision-making in Parkinson's disease. Here we addressed these questions by asking patients with Parkinson's disease (n=26) both on and off their dopamine replacement medication and age- and education-matched healthy controls (n=26) to complete a task designed to isolate the contributions of incremental learning and episodic memory to reward-based learning and decision-making. We found that Parkinson's patients performed as well as healthy controls when using episodic memory, but were impaired at incremental reward-based learning. Dopamine replacement medication remediated this deficit while enhancing subsequent episodic memory for the value of motivationally relevant stimuli. These results demonstrate that Parkinson's patients are impaired at learning about reward from trial-and-error when episodic memory is properly controlled for, and that learning based on the value of single experiences remains intact in patients with Parkinson's disease.

Recent findings in animals have challenged the traditional view of the cerebellum solely as the site of motor control, suggesting that the cerebellum may also be important for learning to predict reward from trial-and-error feedback. Yet, evidence for the role of the cerebellum in reward learning in humans is lacking. Moreover, open questions remain about which specific aspects of reward learning the cerebellum may contribute to. Here we address this gap through an investigation of multiple forms of reward learning in individuals with cerebellum dysfunction, represented by cerebellar ataxia cases. Nineteen participants with cerebellar ataxia and 57 age- and sex-matched healthy controls completed two separate tasks that required learning about reward contingencies from trial-and-error. To probe the selectivity of reward learning processes, the tasks differed in their underlying structure: while one task measured incremental reward learning ability alone, the other allowed participants to use an alternative learning strategy based on episodic memory alongside incremental reward learning. We found that individuals with cerebellar ataxia were profoundly impaired at reward learning from trial-and-error feedback on both tasks, but retained the ability to learn to predict reward based on episodic memory. These findings provide evidence from humans for a specific and necessary role for the cerebellum in incremental learning of reward associations based on reinforcement. More broadly, the findings suggest that alongside its role in motor learning, the cerebellum likely operates in concert with the basal ganglia to support reinforcement learning from reward.

A key question in decision-making is how humans arbitrate between competing learning and memory systems to maximize reward. We address this question by probing the balance between the effects, on choice, of incremental trial-and-error learning versus episodic memories of individual events. Although a rich literature has studied incremental learning in isolation, the role of episodic memory in decision-making has only recently drawn focus, and little research disentangles their separate contributions. We hypothesized that the brain arbitrates rationally between these two systems, relying on each in circumstances to which it is most suited, as indicated by uncertainty. We tested this hypothesis by directly contrasting contributions of episodic and incremental influence to decisions, while manipulating the relative uncertainty of incremental learning using a well-established manipulation of reward volatility. Across two large, independent samples of young adults, participants traded these influences off rationally, depending more on episodic information when incremental summaries were more uncertain. These results support the proposal that the brain optimizes the balance between different forms of learning and memory according to their relative uncertainties and elucidate the circumstances under which episodic memory informs decisions.

Mathematical knowledge is constructed hierarchically during development from a basic understanding of addition and subtraction, two foundational and inter-related, but semantically distinct, numerical operations. Early in development, children show remarkable variability in their numerical problem-solving skills and difficulties in solving even simple addition and subtraction problems are a hallmark of math learning difficulties. Here, we use novel quantitative analyses to investigate whether less distinct representations are associated with poor problem-solving abilities in children during the early stages of math-skill acquisition. Crucially, we leverage dimensional and categorical analyses to identify linear and nonlinear neurobehavioral profiles of individual differences in math skills. Behaviorally, performance on the two different numerical operations was less differentiated in children with low math abilities, and lower problem-solving efficiency stemmed from weak evidence-accumulation during problem-solving. Children with low numerical abilities also showed less differentiated neural representations between addition and subtraction operations in multiple cortical areas, including the fusiform gyrus, intraparietal sulcus, anterior temporal cortex and insula. Furthermore, analysis of multi-regional neural representation patterns revealed significantly higher network similarity and aberrant integration of representations within a fusiform gyrus-intraparietal sulcus pathway important for manipulation of numerical quantity. These findings identify the lack of distinct neural representations as a novel neurobiological feature of individual differences in children's numerical problem-solving abilities, and an early developmental biomarker of low math skills. More generally, our approach combining dimensional and categorical analyses overcomes pitfalls associated with the use of arbitrary cutoffs for probing neurobehavioral profiles of individual differences in math abilities.

Heterogeneity in cognitive and academic abilities is a prominent feature of autism spectrum disorder (ASD), yet little is known about its underlying causes. Here we combine functional brain imaging during numerical problem-solving with hierarchical drift-diffusion models of behavior and standardized measures of numerical abilities to investigate neural mechanisms underlying cognitive variability in children with ASD, and their IQ-matched Typically Developing (TD) peers. Although the two groups showed similar levels of brain activation, the relation to individual abilities differed markedly in ventral temporal-occipital, parietal and prefrontal regions important for numerical cognition: children with ASD showed a positive correlation between functional brain activation and numerical abilities, whereas TD children showed the opposite pattern. Despite similar accuracy and response times, decision thresholds were significantly higher in the ASD group, suggesting greater evidence required for problem-solving. Critically, the relationship between individual abilities and engagement of prefrontal control systems anchored in the anterior insula was differentially moderated by decision threshold in subgroups of children with ASD. Our findings uncover novel cognitive and neural sources of variability in academically-relevant cognitive skills in ASD and suggest that multilevel measures and latent decision-making dynamics can aid in characterization of cognitive variability and heterogeneity in neurodevelopmental disorders.

Human cognition is influenced not only by external task demands but also latent mental processes and brain states that change over time. Here, we use novel Bayesian switching dynamical systems algorithm to identify hidden brain states and determine that these states are only weakly aligned with external task conditions. We compute state transition probabilities and demonstrate how dynamic transitions between hidden states allow flexible reconfiguration of functional brain circuits. Crucially, we identify latent transient brain states and dynamic functional circuits that are optimal for cognition and show that failure to engage these states in a timely manner is associated with poorer task performance and weaker decision-making dynamics. We replicate findings in a large sample (N = 122) and reveal a robust link between cognition and flexible latent brain state dynamics. Our study demonstrates the power of switching dynamical systems models for investigating hidden dynamic brain states and functional interactions underlying human cognition.

Spatial context has been central to theories of memory formation for decades, and it has been more recently suggested that an individual's spatial environment provides a base on which other components of a memory are built ([Robin et al., 2016][1]). Perhaps one reason for the initial focus on space

Little is currently known about dynamic brain networks involved in high-level cognition and their ontological basis. Here we develop a novel Variational Bayesian Hidden Markov Model (VB-HMM) to investigate dynamic temporal properties of interactions between salience (SN), default mode (DMN), and central executive (CEN) networks—three brain systems that play a critical role in human cognition. In contrast to conventional models, VB-HMM revealed multiple short-lived states characterized by rapid switching and transient connectivity between SN, CEN, and DMN. Furthermore, the three “static” networks occurred in a segregated state only intermittently. Findings were replicated in two adult cohorts from the Human Connectome Project. VB-HMM further revealed immature dynamic interactions between SN, CEN, and DMN in children, characterized by higher mean lifetimes in individual states, reduced switching probability between states and less differentiated connectivity across states. Our computational techniques provide new insights into human brain network dynamics and its maturation with development.