Decisions are often thought of as an intermediary between perception and action, but the degree to which this assumption is integrated into different parts of cognitive neuroscience theory and practice varies. After examining these variations on the causal relationship between decisions and actions, this perspective will argue for the claim that decisions and decision processes do not cause actions. An argument will be made that, in place of decision processes, actions are caused by sensorimotor processes. Lastly, ideas are given for studying the sensorimotor processes involved in decisions and actions. The main recommendation is a move to more ecological testing environments that give participants agency over their actions and allow them to learn by continuously updating sensorimotor processes through active sensing.

Hippocampal representations of space and time seem to share a common coding scheme characterized by neurons with bell-shaped tuning curves called place and time cells. The properties of the tuning curves are consistent with Weber’s law, such that, in the absence of visual inputs, width scales with the peak time for time cells and with distance for place cells. Building on earlier computational work, we examined how neurons with such properties can emerge through self-supervised learning. We found that a network based on autoencoders can, given a particular inputs and connectivity constraints, produce scale-invariant time cells. When the animal’s velocity modulates the decay rate of the leaky integrators, the same network gives rise to scale-invariant place cells. Importantly, this is not the case when velocity is fed as a direct input to the leaky integrators, implying that weight modulation by velocity might be critical for developing scale-invariant spatial receptive fields. Finally, we demonstrated that after training, scale-invariant place cells emerge in environments larger than those used during training. Taken together, these findings bring us closer to understanding the emergence of neurons with bell-shaped tuning curves in the hippocampus and highlight the critical role of velocity modulation in the formation of scale-invariant place cells.

Objects associated with both reward and threat produce approach-avoidance conflict (AAC). Although our day-to-day encounters with AAC objects are dynamic and interactive, the cognitive neuroscience literature on AAC is largely based on experiments that use static stimuli. Here, we used a dynamic, interactive, video-game environment to test neural substrates implicated in processing AAC in a more ecologically valid setting. While undergoing functional magnetic resonance imaging (fMRI), subjects (N = 31) played a predator-prey video game, guiding an avatar through a maze containing six types of aversive or appetitive agents. Of the six agent types, two were “non-AAC” and either always healed or always harmed the player’s avatar on contact. The other four were “AAC,” healing or harming the avatar probabilistically. Results revealed that imminence (inverse of distance) between a player’s avatar and an environmental agent was a strong predictor of activation in three brain networks: the cinguloinsular (CI), dorsal frontoparietal (DFP), and occipitotemporal (OT). Additionally, two distinct temporal patterns of heightened activation with AAC agents emerged in two networks: the CI network responded with a transient spike of activation at trial onsets, followed by rapid decay, whereas the lateral frontoparietal (LFP) network showed sustained activation across the whole trial. We conclude that, in an interactive, dynamic setting, the roles of the CI and LFP networks appear to be complimentary, with the CI involved in distinguishing between AAC and non-AAC agents when they first appeared and the LFP involved in maintaining a behavioral mode related to the level of AAC.

Background: Properties of functional connectivity (FC), such as network integration and segregation, are shown to be associated with various human behaviors. For example, Godwin et al. and Sun et al. found increased integration with attention allocation, whereas Cohen and D'Esposito and Shine et al. observed increased segregation with simple motor tasks. The current study investigated how viewing video clips with different valence and arousal influenced integration-segregation properties in task-based FC networks.Methods: We analyzed an open dataset collected by Kim et al. We performed a generalized psychophysiological interaction (gPPI) analysis paired with network analysis and community detection to investigate changes in brain network dynamics when people watched four types of videos that differed by affective valence (unpleasant or pleasant) and arousal (arousing or calm).Results: Results showed that unpleasant arousing videos produced greater FC deviation from the baseline (task-induced FC deviation [tiFCd]) and perturbed the brain into a more segregated state than other kinds of video. Increased segregation was only observed in association systems, not sensorimotor systems.Discussion: Unpleasant arousing content perturbed the brain to a functionally distinct state from the other three types of affective videos. We suggest that the change in brain state was related to people disengaging from the unpleasant arousing content or, alternatively, staying alert while exposed to unpleasant arousing stimuli. The study also added to our understanding of how combining task-based gPPI analysis with community detection methods and network segregation measures can advance our knowledge of the links between behavior and brain state changes.Impact statement Network integration and segregation is an important property of the human brain. We address the question of how affective stimuli influence brain dynamics from a functional connectivity (FC) network integration-segregation perspective. By conducting a whole-brain generalized psychophysiological interaction (gPPI) analysis paired with community detection methods, we found that highly aversive video content induced significant FC changes and perturbed the brain to a more segregated state.

Studies of affective neuroscience have typically employed highly controlled, static experimental paradigms to investigate the neural underpinnings of threat and reward processing in the brain. Yet our knowledge of affective processing in more naturalistic settings remains limited. Specifically, affective studies generally examine threat and reward features separately and under brief time periods, despite the fact that in nature organisms are often exposed to the simultaneous presence of threat and reward features for extended periods. To study the neural mechanisms of threat and reward processing under distinct temporal profiles, we created a modified version of the PACMAN game that included these environmental features. We also conducted two automated meta-analyses to compare the findings from our semi-naturalistic paradigm to those from more constrained experiments. Overall, our results revealed a distributed system of regions sensitive to threat imminence and a less distributed system related to reward imminence, both of which exhibited overlap yet neither of which involved the amygdala. Additionally, these systems broadly overlapped with corresponding meta-analyses, with the notable absence of the amygdala in our findings. Together, these findings suggest a shared system for salience processing that reveals a heightened sensitivity toward environmental threats compared to rewards when both are simultaneously present in an environment. The broad correspondence of our findings to meta-analyses, consisting of more tightly controlled paradigms, illustrates how semi-naturalistic studies can corroborate previous findings in the literature while also potentially uncovering novel mechanisms resulting from the nuances and contexts that manifest in such dynamic environments.

Neural and computational evidence suggests that perceptual decisions depend on an evidence accumulation process. The gradual reveal fMRI method, which prolongs a decision to match the slow temporal resolution of fMRI measurements, has classified dorsal visual stream regions as "Action" (alternatively, "Moment of Recognition" or "Commitment") and ventral visual stream regions as "Accumulator." Previous gradual reveal fMRI studies, however, only tested actions that were in response to decisions and, thus, related to evidence accumulation. To fully dissociate the contribution of sensory, decision, and motor components to Action and Accumulator regions in the dorsal and ventral visual streams, we extended the gradual reveal paradigm to also include responses made to cues where no decision was necessary. We found that the lateral occipital cortex in the ventral visual stream showed a highly selective Accumulator profile, whereas regions in the fusiform gyrus were influenced by action generation. Dorsal visual stream regions showed strikingly similar profiles as classical motor regions and also as regions of the salience network. These results suggest that the dorsal and ventral visual streams may appear highly segregated because they include a small number of regions that are highly selective for Accumulator or Action. However, the streams may be more integrated than previously thought and this integration may be accomplished by regions with graded responses that are less selective (i.e., more distributed).

Abstract Introduction Many theories contend that evidence accumulation is a critical component of decision‐making. Cognitive accumulation models typically interpret two main parameters: a drift rate and decision threshold. The former is the rate of accumulation, based on the quality of evidence, and the latter is the amount of evidence required for a decision. Some studies have found neural signals that mimic evidence accumulators and can be described by the two parameters. However, few studies have related these neural parameters to experimental manipulations of sensory data or memory representations. Here, we investigated the influence of affective salience on neural accumulation parameters. High affective salience has been repeatedly shown to influence decision‐making, yet its effect on neural evidence accumulation has been unexamined. Methods The current study used a two‐choice object categorization task of body images (feet or hands). Half the images in each category were high in affective salience because they contained highly aversive features (gore and mutilation). To study such quick categorization decisions with a relatively slow technique like functional magnetic resonance imaging, we used a gradual reveal paradigm to lengthen cognitive processing time through the gradual “unmasking” of stimuli. Results Because the aversive features were task‐irrelevant, high affective salience produced a distractor effect, slowing decision time. In visual accumulation regions of interest, high affective salience produced a longer time to peak activation. Unexpectedly, the later peak appeared to be the product of changes to both drift rate and decision threshold. The drift rate for high affective salience was shallower, and the decision threshold was greater. To our knowledge, this is the first demonstration of an experimental manipulation of sensory data or memory representations that changed the neural decision threshold. Conclusion These findings advance our knowledge of the neural mechanisms underlying affective responses in general and the influence of high affective salience on object representations and categorization decisions.

Previous literature has shown that the involvement of the lateral occipital complex (LOC) in object recognition may operate via a mechanism of evidence accumulation. This work also has shown that these accumulators mimic cognitive diffusion models, specifically in that object characteristics which affect recognition time also affect accumulation slopes (James et al., 2006; Ploran et. al., 2007; Ratcliff et. al., 2016). Perceptual experience with an object (via priming/adaptation) also influenced accumulation slope (James et al., 2000). Another prominent attribute of objects that may affect accumulation is the visual salience. Visual salience affects sensory processing typically by orienting attention more efficiently, but how this affects visual evidence accumulators in the brain has not been investigated (Itti 2007). Here, we used a gradual reveal paradigm, which slowed visual information presentation, paired with fMRI to measure the BOLD response to color-saturated (salient) and non-saturated (non-salient) images while participants completed a 2-alternative-forced-choice image recognition task (n=17). Consistent with our hypothesis, salient objects were recognized faster and peak BOLD activation occurred sooner. However, the slower peak for non-salient objects was not due to a steeper accumulation slope or higher threshold. Instead, non-salient objects showed a delayed onset of accumulation, after which accumulation proceeded with the same slope and to the same threshold as the salient objects. The results add to our knowledge of how accumulation of evidence contributes to object recognition in the brain, specifically in areas considered visual evidence accumulators, such as the LOC. The results suggest that the speed advantage conveyed by higher salience involves the initiation of evidence accumulation, rather than changes to the accumulation mechanism itself. This suggests that the neural mechanism underlying orienting responses may be a more rapid accumulation onset.

Previous literature suggests that visual regions of the brain such as the lateral occipital cortex and fusiform gyrus are involved in object recognition and may operate as perceptual evidence accumulators (James et al., 2000; Ploran et al., 2007). This work also suggests that these accumulators mimic the properties of cognitive models of evidence accumulation (or, more generally, diffusion models) that have been used to formally assess the cognitive mechanisms of decision making (James et al., 2006; Ratcliff & Smith, 2004). That is, changes to object characteristics have predictable effects on reaction time and neural accumulation slopes. An important characteristic of objects that may affect evidence accumulation is affective salience, which is known to affect object recognition. Here, we used a gradual reveal fMRI paradigm, which slows the presentation of visual information, to assess the influence of affective salience on the accumulation of evidence leading to recognition. Subjects (n=37) performed a 2-alternative forced-choice decision task on categories of body parts that had either high negative affective salience(body parts with mutilations, lacerations) or low affective salience. As expected, high salience produced longer reaction times (distractor effect) during object recognition and were accompanied by longer time to peak activation (evidence). More unexpected was the lack of significant difference between the accumulation slopes (drift rates) of the salient and non-salient conditions. Thus, affective salience did not affect the mechanism responsible for accumulation of evidence. Instead, high affective salience appeared to raise the threshold of neural evidence needed for a decision. The results suggest that highly salient, distracting attributes of objects may raise neural decision thresholds.

Similar to how attention can enhance or suppress a visual target in a specific location in space, attention can also tune the perception of a visual event at a specific moment in time. One way to observe this temporal tuning is to present an auditory stimulus aligned with a visual target event. In general, past research reported that a co-occurring auditory stimulus decreases the amplitude of the N1 component of the visual event-related potential (ERP), possibly suppressing bottom-up attention evoked by a visual target. However, an independent line of research found that an expected auditory stimulus, such as one embedded in a rhythmic pattern, increases N1 amplitude to a coincident visual target event. Here, we present a novel preregistered demonstration of this dissociation using the flash-lag paradigm. Specifically, we observe opposite effects of rhythmic (expected) and non-rhythmic (unexpected) sounds on the visual ERP to a coincident flash. Results suggest that expectation for an upcoming sound reverses the sound’s typical suppression of bottom-up visual attention. We discuss these findings in the context of the multisensory tuning of attention.

Rationale The co-occurrence of alcohol consumption and sexual activity is associated with increased risk for sexual assault, sexually transmitted disease, and unplanned pregnancy among young adult women with alcohol use disorder (AUD). There is considerable previous work demonstrating neural reactivity to alcohol cues in AUD. Because alcohol consumption and sexual behavior are both rewarding and tend to co-occur, sexual cues may produce similar neural reactivity in women with AUD, possibly indicating a shared mechanism underlying reactivity to both types of cues. Alternatively, reactivity to alcohol versus sexual cues may be distinct, suggesting domain-specific mechanisms. Objectives We investigated whether the decision vulnerabilities in AUD women regarding sexual activity were related to differences in brain activation compared to control women. Methods Women with ( n = 15) and without ( n = 16) AUD completed a hypothetical decision-making task during fMRI that presented low- or high-risk scenarios involving visual sexual, appetitive, and neutral cues. Results Results showed that sexual cues were more often endorsed by women with AUD compared to controls and elicited differential brain activation patterns in frontal, visual, and reward regions. During high-risk decisions, women with AUD failed to downregulate activation, causing hyperactivation compared to controls. Conclusions Visual sexual cues produced reactivity like that previously demonstrated for alcohol cues, suggesting a shared or domain-general mechanism for alcohol and sexual cue reactivity in women with AUD. Riskier sexual decisions in women with AUD may be a consequence of repeatedly pairing alcohol use and sexual activity, a characteristic behavior of this population.

The two visual streams hypothesis is a robust example of neural functional specialization that has inspired countless studies over the past four decades. According to one prominent version of the theory, the fundamental goal of the dorsal visual pathway is the transformation of retinal information for visually-guided motor behavior. To that end, the dorsal stream processes input using absolute (or veridical) metrics only when the movement is initiated, necessitating very little, or no, memory. Conversely, because the ventral visual pathway does not involve motor behavior (its output does not influence the real world), the ventral stream processes input using relative (or illusory) metrics and can accumulate or integrate sensory evidence over long time constants, which provides a substantial capacity for memory. In this study, we tested these relations between functional specialization, processing metrics, and memory by training identical recurrent neural networks to perform either a viewpoint-invariant object classification task or an orientation/size determination task. The former task relies on relative metrics, benefits from accumulating sensory evidence, and is usually attributed to the ventral stream. The latter task relies on absolute metrics, can be computed accurately in the moment, and is usually attributed to the dorsal stream. To quantify the amount of memory required for each task, we chose two types of neural network models. Using a long-short-term memory (LSTM) recurrent network, we found that viewpoint-invariant object categorization (object task) required a longer memory than orientation/size determination (orientation task). Additionally, to dissect this memory effect, we considered factors that contributed to longer memory in object tasks. First, we used two different sets of objects, one with self-occlusion of features and one without. Second, we defined object classes either strictly by visual feature similarity or (more liberally) by semantic label. The models required greater memory when features were self-occluded and when object classes were defined by visual feature similarity, showing that self-occlusion and visual similarity among object task samples are contributing to having a long memory. The same set of tasks modeled using modified leaky-integrator echo state recurrent networks (LiESN), however, did not replicate the results, except under some conditions. This may be because LiESNs cannot perform fine-grained memory adjustments due to their network-wide memory coefficient and fixed recurrent weights. In sum, the LSTM simulations suggest that longer memory is advantageous for performing viewpoint-invariant object classification (a putative ventral stream function) because it allows for interpolation of features across viewpoints. The results further suggest that orientation/size determination (a putative dorsal stream function) does not benefit from longer memory. These findings are consistent with the two visual streams theory of functional specialization.

Using visual information to perform actions is a fundamental aspect of human behavior. Musicians commonly translate visual information into action using both concrete and abstract visual information. We exposed expert guitarists to four types of familiar visual depictions of action instruction including musical notation (very abstract), tablature (abstract), chord diagrams (more concrete), and actual pictures of guitars chords being formed (very concrete). These were shown during fMRI scanning as the guitarists formed the appropriate chords (as visually depicted) on a magnet safe guitar fret board with strings, or where they simply viewed the visual stimuli without an action. Whole brain contrasts revealed that the right precuneus was more active for abstract instruction while an occipito-insular circuit was more active for concrete instruction. The current findings highlight that the degree of over-learned visual abstraction is an important factor modulating visual-motor processing.

Letter production relies on a tight coupling between motor movements and visual feedback-each stroke of the letter is visually experienced as it is produced. Experience with letter production leads to increases in functional connectivity, a measure of neural communication, among visual and motor brain systems and leads to gains in letter recognition in preliterate children. We hypothesized that the contingency between the motor and visual experiences of the written form during production would result in both effects. Twenty literate adults were trained on four sets of novel symbols over the course of one week. Each symbol set was trained through one of four training conditions: drawing with ink, drawing without ink, watching a handwritten symbol unfold as if being drawn, and watching a static handwritten symbol. Contingency of motor and visual experiences occurred in the drawing with ink condition. The motor and visual experiences were rendered non-contingent in each of the other three conditions by controlling for visual or motor experience. Participants were presented with the trained symbols during fMRI scanning at three time points: one pre-training, one post-training, and one after a week-long no-training delay. Recognition was tested after each training session and after the third scan. We found that the contingency between visual and motor experiences during production changed the pattern of functional connectivity among visual, motor, and auditory neural communities and resulted in better recognition performance at post-training than at pre-training. Recognition gains were maintained after the no-training delay, but the functional connections observed immediately after training returned to their pre-training baselines. Our results suggest that behaviors that couple sensory and motor systems result in temporary changes in neural communication during perception that may not directly support changes in recognition.

Music is a ubiquitous feature of young adults' social drinking environments, yet no studies have assessed whether and how it impacts risky decisions to drink alcohol. Previous research on the influence of music on risky decisions is largely based around decision tasks with monetary incentives. Methods: To assess the impact of music listening on risky drinking decisions, the current study used visual alcohol cues paired with hypothetical risky drinking scenarios (e.g., "You do not have a safe ride home" for alcohol). Young adult women with a history of alcohol abuse (N = 34) and casual-drinking control women (N = 29) made hypothetical decisions about whether or not to drink alcohol, or eat food (an appetitive control condition), in risky contexts while personal "party music" (music chosen by participants for "going out") and "home music" (music chosen for "staying in") played in the background. The main dependent measure e likelihood of drinking e was reported on a 4-point scale where 1 corresponded to "very unlikely", and 4 to "very likely". Results: Listening to party music while making decisions increased the likelihood of making risky decisions, regardless of alcohol abuse history, while other personal music did not. Further, party music specifically increased the likelihood of risky drinking decisions relative to risky eating decisions. As expected, those with a history of alcohol abuse made more risky drinking decisions in general, regardless of the type of music heard. Discussion: The results suggest that party music is an important feature of the drinking environment associated with increased risky decisions about drinking alcohol in young adult women, regardless of their history of alcohol abuse. The finding that music plays an important role in risky drinking decisions indicates that further investigation into the real-world drinking environments of young adults is crucial, as it will aid in the development of a more complete picture of risky drinking decisions in young adults. (C) 2019 Elsevier Inc. All rights reserved.

The centuries-old philosophical question posed by William Molyneux and referred to by John Locke (1689) concerns the transfer of object information from touch to vision. Solvi, Al-Khudhairy, and Chittka (2020) tested this question comparatively in bumble bees and found evidence of crossmodal object recognition between touch and vision.

Externalizing psychopathology (EXT) is characterized by poor decision-making in situations that involve simultaneous cues for approach and avoidance behavior (i.e. approach-avoidance conflicts). Previous studies of EXT have examined these deficits primarily using tasks involving decisions between positive reward and negative punishment, suggesting that EXT is characterized by a general bias towards high salience (e.g. temporally proximal or reward) cues relative to low salience (e.g. temporally distal or loss) cues. However, in order to better characterize decision-making in approach-avoidance conflicts, the present study utilized a novel task to examine neural activation in contexts involving both positive reward and negative punishment as well as positive punishment and negative reward by manipulating physical proximity of affective cues. Neuroimaging results indicated that EXT was associated with deficits related to cue prioritization based on salience, suggesting that failure to differentiate relevant from less relevant information contributes to poor decision-making among individuals with EXT.

Perceptual history – learning from perceptual experiences (Jeong et al, 2014) – influences item salience and processing efficiency (Kahnt & Tobler, 2017; James et al. 2000). From the addiction literature, it’s found that an addiction-related cue acquires increased neural reactivity in sensory cortex (Hanlon et al., 2014). We hypothesized that repeat exposure to addictive substances induces sensory system plasticity leading to increased salience and sensory processing speed of substance-related items(Robinson & Berridge 2008; Yalechkov et al., 2010). We selected an addictive substance that a majority of the American population engages with: alcohol (Grant et al., 2017). Participants chose three drink types - preferred, aversive, and neutral - from 16 alcohol categories. We predicted that preferred drinks would show faster processing speed because of their enhanced perceptual history. In three experiments, we measured performance with sensory processing efficiency tasks. Experiments 1 and 2 consisted of oddball visual search (VS), where successful localization of the target required processing all images displayed. In homogeneous VS (Exp. 1, n=32), participants searched for the unique image in a visual display of many identical images. In heterogenous VS (Exp. 2, n=31) participants searched for the unique category image (target) in a display of varied images from a distractor category. In task three, multiple object tracking (Exp. 3, n=33), participants visually tracked a subset of moving alcoholic beverage images. In Experiment 1, displays with preferred items had shortest reaction times. In Experiment 2, the same preferred item facilitation was found. In Experiment 3, accuracy was highest when tracking preferred items, while preferred distractors reliably decreased accuracy indicating that processing preferred items was automatic. Overall, task performance was enhanced with task-relevant preferred items and hindered with task-irrelevant. The coupling of perceptual history (preference) with high reward (addictive substances) leads to neuro-plasticity observable in measures of sensory salience.