We tend to interpret feedback in ways that confirm our pre-existing beliefs. Such confirmatory tendencies are often viewed as cognitive flaws, but might have adaptive facets. We propose a novel experimental paradigm and Bayesian computational model to examine how confirmatory inference shapes reward learning when outcomes have ambiguous valence. In these cases, interpretation involves integrating prior beliefs with incomplete evidence, reflecting an inductive bias analogous to missing data imputation. We develop and test this theory using a reward learning task in which information about outcome valence (but not magnitude) is sometimes withheld, allowing for subjective interpretation. Our Bayesian model explains the dynamics of behavior and stated beliefs better than alternative reinforcement learning models. Moreover, stated beliefs about the positivity of ambiguous outcomes are correlated with optimism. Together, these findings demonstrate how confirmatory reward learning can emerge from inference under ambiguity, and may be individually linked to broader dispositional traits.

People constantly invest effort to seek information in everyday life, even if the information is of no instrumental value for the decision at hand. However, the mechanism underlying people’s willingness to exert effort in exchange for information remains unclear. Previous work has suggested that desirability and uncertainty are two key factors influencing information-seeking, but few studies have tested if these factors are important when exerting physical effort in exchange for information. Additionally, it remains unknown whether different kinds of uncertainty, i.e., risk and ambiguity, have differential impacts. In the current study, we developed a novel information-seeking task to quantify the value of non-instrumental information using incentive-compatible physical effort to examine how effort changes as a function of desirability and uncertainty. Across two studies (N = 123), we found that people’s willingness to exert effort is positively associated with outcome expected value under both risk and ambiguity. Additionally, people exert more effort when outcome distribution uncertainty increases in risky situations, but are insensitive to ambiguity, except when facing extreme ambiguity. Our results demonstrate an unexpected dissociation. Humans will engage in effort-based information-seeking, even for non-instrumental information, when facing risk. In contrast, they show a much lower willingness to expend effort to resolve non-instrumental ambiguity.

Agency beliefs influence how humans learn from different contexts and outcomes. Research demonstrates that stressors, such as exposure to early-life adversity (ELA), are associated with both agency beliefs and learning, but how these processes interact remains unclear. The current study investigated whether exposure to ELA influences agency and interacts with reinforcement learning in adults. Replicating prior behavioral and computational work, ELA resulted in decreased learning, while increased adversity severity was associated with decreased latent agency beliefs. These findings suggest that exposure to adversity in childhood has a nuanced impact on reinforcement learning and agency beliefs in adulthood.

When recalling autobiographical events, people retrieve not only the event details, but also the feelings they experienced. Past work with different measures of memories for feelings remain inconclusive, suggesting that people are either highly consistent or inconsistent with remembering feelings. The current study examined whether people are able to consistently recall the intensity of previous feelings associated with consequential and negatively valenced emotional events, i.e., the 9/11 attack (N = 769) and Covid-19 pandemic (N = 726). By comparing the initial and recalled intensities of negative feelings, we found that people systematically recall more intense negative feelings than they initially reported – overestimating the intensity of past negative emotional experience. The Covid-19 dataset further showed that people whose emotional well-being improved more demonstrate smaller biases in remembered feelings. Across both datasets, the remembered intensity of feelings correlated with initial feelings and were also influenced by current feelings, although the impact of the current feelings was stronger in the Covid-19 dataset than the 9/11 dataset. Our results suggest that when recalling negative autobiographical events, people tend to overestimate the intensity of experienced negative emotional experience with the degree of bias influenced by current feelings and well-being.

Attributing outcomes to your own actions or to external causes is essential for appropriately learning which actions lead to reward and which actions do not. Our previous work showed that this type of credit assignment is best explained by a Bayesian reinforcement learning model which posits that beliefs about the causal structure of the environment modulate reward prediction errors (RPEs) during action value updating. In this study, we investigated the brain networks underlying reinforcement learning that are influenced by causal beliefs using functional magnetic resonance imaging while human participants (n = 31; 13 males, 18 females) completed a behavioral task that manipulated beliefs about causal structure. We found evidence that RPEs modulated by causal beliefs are represented in dorsal striatum, while standard (unmodulated) RPEs are represented in ventral striatum. Further analyses revealed that beliefs about causal structure are represented in anterior insula and inferior frontal gyrus. Finally, structural equation modeling revealed effective connectivity from anterior insula to dorsal striatum. Together, these results are consistent with a possible neural architecture in which causal beliefs in anterior insula are integrated with prediction error signals in dorsal striatum to update action values.

Beliefs about the controllability of positive or negative events in the environment can shape learning throughout the lifespan. Previous research has shown that adults’ learning is modulated by beliefs about the causal structure of the environment such that they update their value estimates to a lesser extent when the outcomes can be attributed to hidden causes. This study examined whether external causes similarly influenced outcome attributions and learning across development. Ninety participants, ages 7 to 25 years, completed a reinforcement learning task in which they chose between two options with fixed reward probabilities. Choices were made in three distinct environments in which different hidden agents occasionally intervened to generate positive, negative, or random outcomes. Participants’ beliefs about hidden-agent intervention aligned with the true probabilities of the positive, negative, or random outcome manipulation in each of the three environments. Computational modeling of the learning data revealed that while the choices made by both adults (ages 18–25) and adolescents (ages 13–17) were best fit by Bayesian reinforcement learning models that incorporate beliefs about hidden-agent intervention, those of children (ages 7–12) were best fit by a one learning rate model that updates value estimates based on choice outcomes alone. Together, these results suggest that while children demonstrate explicit awareness of the causal structure of the task environment, they do not implicitly use beliefs about the causal structure of the environment to guide reinforcement learning in the same manner as adolescents and adults.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

People learn differently from good and bad outcomes. We argue that valence-dependent learning asymmetries are partly driven by beliefs about the causal structure of the environment. If hidden causes can intervene to generate bad (or good) outcomes, then a rational observer will assign blame (or credit) to these hidden causes, rather than to the stable outcome distribution. Thus, a rational observer should learn less from bad outcomes when they are likely to have been generated by a hidden cause, and this pattern should reverse when hidden causes are likely to generate good outcomes. To test this hypothesis, we conducted two experiments ( N = 80, N = 255) in which we explicitly manipulated the behavior of hidden agents. This gave rise to both kinds of learning asymmetries in the same paradigm, as predicted by a novel Bayesian model. These results provide a mechanistic framework for understanding how causal attributions contribute to biased learning.

A Pavlovian bias to approach reward-predictive cues and avoid punishment-predictive cues can conflict with instrumentally-optimal actions. Here, we propose that the brain arbitrates between Pavlovian and instrumental control by inferring which is a better predictor of reward. The instrumental predictor is more flexible; it can learn values that depend on both stimuli and actions, whereas the Pavlovian predictor learns values that depend only on stimuli. The arbitration theory predicts that the Pavlovian predictor will be favored when rewards are relatively uncontrollable, because the additional flexibility of the instrumental predictor is not useful. Consistent with this hypothesis, we find that the Pavlovian approach bias is stronger under low control compared to high control contexts.

Behavioral evidence suggests that beliefs about causal structure constrain associative learning, determining which stimuli can enter into association, as well as the functional form of that association. Bayesian learning theory provides one mechanism by which structural beliefs can be acquired from experience, but the neural basis of this mechanism is poorly understood. We studied this question with a combination of behavioral, computational, and neuroimaging techniques. Male and female human subjects learned to predict an outcome based on cue and context stimuli while being scanned using fMRI. Using a model-based analysis of the fMRI data, we show that structure learning signals are encoded in posterior parietal cortex, lateral prefrontal cortex, and the frontal pole. These structure learning signals are distinct from associative learning signals. Moreover, representational similarity analysis and information mapping revealed that the multivariate patterns of activity in posterior parietal cortex and anterior insula encode the full posterior distribution over causal structures. Variability in the encoding of the posterior across subjects predicted variability in their subsequent behavioral performance. These results provide evidence for a neural architecture in which structure learning guides the formation of associations. SIGNIFICANCE STATEMENT Animals are able to infer the hidden structure behind causal relations between stimuli in the environment, allowing them to generalize this knowledge to stimuli they have never experienced before. A recently published computational model based on this idea provided a parsimonious account of a wide range of phenomena reported in the animal learning literature, suggesting a dedicated neural mechanism for learning this hidden structure. Here, we validate this model by measuring brain activity during a task that involves both structure learning and associative learning. We show that a distinct network of regions supports structure learning and that the neural signal corresponding to beliefs about structure predicts future behavioral performance.

Psychopathy is a personality disorder with strong links to criminal behavior. While research on psychopathy has focused largely on socio-affective dysfunction, recent data suggest that aberrant decision making may also play an important role. Yet, the circuit-level mechanisms underlying maladaptive decision making in psychopathy remain unclear. Here, we used a multi-modality functional imaging approach to identify these mechanisms in a population of adult male incarcerated offenders. Psychopathy was associated with stronger subjective value-related activity within the nucleus accumbens (NAcc) during inter-temporal choice and with weaker intrinsic functional connectivity between NAcc and ventromedial prefrontal cortex (vmPFC). NAcc-vmPFC connectivity strength was negatively correlated with NAcc subjective value-related activity; however, this putative regulatory pattern was abolished as psychopathy severity increased. Finally, weaker cortico-striatal regulation predicted more frequent criminal convictions. These data suggest that cortico-striatal circuit dysregulation drives maladaptive decision making in psychopathy, supporting the notion that reward system dysfunction comprises an important neurobiological risk factor.

Antisociality is commonly conceptualized as a unitary construct, but there is considerable evidence for multidimensionality. In particular, two partially dissociable symptom clusters-psychopathy and externalizing-have divergent associations to clinical and forensic outcomes and are linked to unique patterns executive dysfunction. Here, we used fMRI in a sample of incarcerated offenders to map these dimensions of antisocial behavior to brain circuits underlying two aspects of inhibitory self-control: interference suppression and response inhibition. We found that psychopathy and externalizing are characterized by unique and task-selective patterns of dysfunction. Although higher levels of psychopathy predicted increased activity within a distributed frontoparietal network for interference suppression, externalizing did not predict brain activity during attentional control. By contrast, each dimension had opposite associations to frontoparietal activity during response inhibition. These findings provide neurobiological evidence supporting the fractionation of antisocial behavior and identify dissociable mechanisms through which different facets predispose dysfunction and impairment.

Criminal behavior is a costly social problem, making it a crucial subject for scientific investigation. While the risk architecture underlying criminality is complex, psychopathy has emerged as a particularly robust predictor of criminal behavior and recidivism. Psychopathy is a personality disorder characterized by a combination of superficial charm, persistent instrumental antisocial behavior, marked sensation-seeking and impulsivity, blunted empathy and punishment sensitivity, and shallow emotion. While there is a wealth of data suggesting that many of the phenotypical attributes of the disorder are heritable, few specific genetic markers have been identified, and the etiopathophysiology of psychopathy remains largely unknown. This chapter elucidates possible genetic and neurobiological mechanisms for psychopathy and antisocial behavior, clarifies the current limitations of genetics research in this area, and outlines necessary future directions for this field.

Antisocial behavior is an enormously costly social problem, but its origins are poorly understood. A new study shows that prosocial and antisocial behaviors arise from individual differences in how we represent the value of others’ pain relative to our own potential gain, rather than from variability in the capacity for effortful inhibitory control.

Aggression may be present across a large part of the spectrum of psychopathology, and underlies costly criminal antisocial behaviors. Human aggression is a complex and underspecified construct, confounding scientific discovery. Nevertheless, some biologically tractable subtypes are apparent, and one in particular-impulsive (reactive) aggression-appears to account for many facets of aggression-related dysfunction in psychiatric illness. Impulsive-aggression is significantly heritable, suggesting genetic transmission. However, the specific neurobiological mechanisms that mediate genetic risk for impulsive-aggression remain unclear. Here, we review extant data on the genetics and neurobiology of individual differences in impulsive-aggression, with particular attention to the role of genetic variation in Monoamine Oxidase A (MAOA) and its impact on serotonergic signaling within corticolimbic circuitry.