To interact efficiently with our environment, our brain predicts the sensory effects of our actions and compares them with the actual outcomes. This allows us to adapt our actions when predictions and sensory outcomes mismatch. While this process is generally well understood for action-sound predictions, it is an open question whether these predictions can flexibly switch in frequently changing environments, as they occur in real life. To investigate the flexibility of top-down predictions, we asked participants (N = 41) to press one of two buttons, a left-hand and a right-hand button, and switch hands autonomously. One button frequently produced a sound (80%) and rarely no sound. The other button frequently generated no sound (80%) and rarely produced a sound. In a third, separate condition, each button produced a sound in 50% of the trials. Unexpected sounds and unexpected sound omissions elicited a series of error-related brain responses in the electroencephalogram (EEG) at different levels of auditory processing, including a mismatch negativity (MMN) and the P3 complex for unexpected sounds, and the oN1, oN2, and oP3 complex for unexpected omissions. Moreover, unexpected sounds elicited an equivalent MMN, regardless of whether silence was expected (80%) or no reliable expectation was possible (50%), while later P3 components showed different amplitudes. Our results demonstrate flexible action-sound predictions at sensory and higher cortical levels. Furthermore, they indicate that predicted silence does not have an explicit sensory representation at lower levels but emerges at later stages, when higher-level information has been integrated.
Blocked-cyclic picture naming is a prominent tool in word production research. Responses were slower when the stimuli were presented blocked by semantic category than when they are presented intermixed. Wöhner et al. (2021) extended the task to sound naming and found that the semantic interference was much larger in sound naming (e.g., a barking) than in picture naming (e.g., of a dog), by a factor of approximately three, even when the same words were produced (e.g., “dog”). We explored possible reasons for this differential interference pattern. We replicated the differential interference when the stimuli were presented briefly, ruling out that it results from substantial overlap between stimulus processing on the one hand and phonological response preparation and pre-articulatory self-monitoring on the other. In contrast, we found similar-sized interference when the sounds and pictures were replaced with spoken and written words to which participants responded with the translation in their second language, ruling out that the differential interference results from different input modalities. We also observed the differential interference in sound and picture naming when the spoken response was replaced with a typed response, ruling out that it results from a specific (spoken) response modality. Overall, the data demonstrate the robustness of the differential interference in sound and picture naming and provide evidence that the semantic interference in blocked-cyclic naming is effectively shaped by early stimulus-specific perceptual and semantic – “lead-in” – processes, which precede lexical selection. These early processes require more attention in word production research.
Abstract Attentional control requires the fine-tuned interaction of attentional networks supporting alertness, orienting and higher-level executive control. This study examined the interaction of executive attention, comprising inhibition, and involuntary orienting towards unexpected deviant sounds in children (6–8-years, N=30), adolescents (10–12-years, N=39) and adults (18–34-years, N=35). An auditory equiprobable Go/Nogo-Oddball paradigm was employed to investigate executive (Go/Nogo) and involuntary (Oddball) attention and their interaction. Event-related potentials (ERP) in the EEG, pupil dilation and behavioral data were analyzed using Bayesian statistics. Deviant sounds evoked an involuntary attentional orienting reflected by the ERP component P3a and decreased performance in all age groups. The distraction effect diminished with age. Pupil dilation in response to deviant and target sounds were modulated by involuntary and executive attention but showed no interaction. In the EEG, frontal Nogo-N2 and Nogo-P3 effects, which reflect response inhibition, appeared in standard trials but not in deviant trials. This interaction suggests that response inhibition is reduced by involuntary attention orienting. While all age groups showed similar amplitudes of the Nogo-N2 effect, the subsequent Nogo- P3 effect, which has been associated to motor inhibition, was absent in the 6–8-year-olds. Age differences observed in latencies of the Nogo-N2 effect but not in the latencies of the Nogo- P3 effect (between adolescents and adults) indicate distinct developmental trajectories of the response inhibition mechanisms underlying the Nogo-N2 and Nogo-P3 effects throughout childhood and adolescence. The present data provide novel insights into the interaction between executive and orienting attention networks in the auditory modality during development.
The unpredicted omission of expected sound, that is “silence”, elicits omission N1, N2 and P3 (early and late oN1, oN2, oP3) event-related potential (ERP) and pupil dilation responses (PDR), which are interpreted as prediction error signals located along different processing levels. This study aims to investigate whether these responses to unpredicted silence can also be elicited by the unpredicted continuation of sound, that is, the “absence” of expected silence. A prediction error response to the absence of silence would indicate that silence was predicted including the levels of processing on which a prediction of silence was established. Participants (27 women, 10 men) pressed a button every 1-2s while exposed to continuous Brownian noise. Button presses predictably (88% of trials), randomly (50% of trials) or never (0% of trials; motor-control) interrupted the noise. EEG and pupil diameter data were recorded. We found that the unpredicted absence of a silent interval in the noise elicited late oN1 (~150ms) and oN2, oP3 and PDR omission prediction error responses. However, we did not find an early oN1 (~100ms). Predictive representations of silence can be established at the level of categorical sensory processing (late oN1) and at the levels of later, higher cognitive processing and attention (oN2, oP3, and PDR), but not at the level of feature-based, early cortical sensory processing (early oN1), possibly because the absence of features cannot easily be encoded. These results suggest that silence is represented not only as the absence of sound but also as a predictable event in itself.
The mismatch negativity (MMN) is a well-studied event-related potential (ERP) component in the EEG reflecting deviance detection in the auditory modality. It taps into the basic functioning of auditory regularity processing. The auditory multi-feature paradigm is widely used in sensitive and special populations to measure MMN simultaneously for different sound features in a short amount of time. It is consensus in the field that both adaptation and genuine deviance detection contribute to the “classic” MMN computed as deviant minus standard ERP difference. However, no attempts have yet been made to disentangle adaptation from “genuine” MMN in the multi-feature paradigm. Here, we propose a cascadic control condition for the auditory multi-feature paradigm that controls for adaptation and physical differences between standard and deviant sounds. Using this new paradigm, we measured genuine MMN, computed as deviant minus control ERP difference, for frequency, location, intensity, and duration deviants. Genuine MMN amplitudes for frequency and location were found substantially smaller than in traditional paradigms. No genuine intensity MMN and only a later and smaller genuine duration MMN were found. The results suggest stronger contributions of adaptation than in the traditional oddball paradigm. Controlling for adaptation is particularly relevant in research concerning predictive processing and the use of MMN as a biomarker related to impaired NMDA receptor synaptic transmission as observed in schizophrenia. The presented multi-feature cascadic control condition enables the measurement of genuine MMN, which presumably reflects higher-order cortical computations, such as predictive processing, still in a short amount of time.
The human brain anticipates the sensory consequences of an action and generates a prediction error (PE) signal when the intended action effect does not occur. This study investigated auditory event-related potential PE markers based on whether participants intended to perform a specific action or produce a specific action effect. Participants were instructed to press a left or right button to produce low- or high-pitched tones, following a visual pattern. The instructions, actions and tone sequences were identical for all participants. The visual patterns differed in two groups: In the action-effect intention group, the visual pattern consisted of 'notes' (indicating low/high pitch). In the action intention group, the visual pattern consisted of 'letters' (indicating left/right button-press). In both groups, a button-press occasionally failed to produce the associated tone (incongruent sounds). The key finding was that these incongruent sounds elicited an enhanced auditory N1 component compared to congruent sounds only in the 'notes' group. We propose that participants in the 'notes' group selected their actions based on the intended action effect, which induced a predictive sensory representation of the expected tone. A violation of this prediction resulted in the early PE, reflected in the auditory N1. Later PE responses, specifically the N2b and P3 components, were observed in both groups. This suggests that action-effect associations were represented, and their violation was processed at a conceptual level even in the 'letters' group. These results support theories postulating that event representations integrate features of stimuli, actions and their associated outcomes.
Predictive coding conceptualizes attention as a weighting of prediction error signals. However, empirical findings on how attention influences common markers of prediction error have been inconsistent, likely because these markers are typically derived from stimulus-evoked responses. To avoid stimulus-related confounds and isolate effects related purely to prediction, we investigated how attention modulates brain responses to unexpected stimulus omissions. Using visual-auditory couplings where the auditory stimulus was occasionally omitted, we recorded EEG responses that revealed a multistage omission response-from early sensory to later higher-level prediction error activity. Voluntary attention was manipulated along two dimensions: (1) toward the visual or auditory modality, and (2) toward the moment of stimulus presentation or sustained over time. Early sensory prediction error, reflected by the omission N1, was unaffected by any manipulation of attention. In contrast, later high-level prediction error processing, reflected by omission P3 responses, was strongly affected by directing attention: robust responses were elicited when attention was directed to the auditory modality-where the prediction had been violated-but these were markedly reduced or absent when attention was directed to the visual modality. These results suggest an attentional system that does not affect low-level sensory prediction error but is capable of influencing distinct stages in the processing hierarchy in service of task performance. This first investigation of how attention affects different stages of omission activity suggests that voluntary attention may modulate prediction error processing via specific neurotransmitter systems and demonstrates this approach's potential for reliably studying precision-weighting in the brain.
Abstract To interact efficiently with our environment, our brain predicts the sensory effects of our actions and compares them with the actual outcomes. This allows us to adapt our actions when predictions and sensory outcomes mismatch. While this process is generally well understood for action-sound predictions, it is an open question how flexibly these predictions can adapt in frequently changing environments, as they occur in real life. To investigate the flexibility of top-down predictions, we asked participants (N = 41) to press one of two buttons, a left-hand and a right-hand button, and switch hands autonomously. One button frequently produced a sound (80%) and rarely no sound. The other button frequently generated no sound (80%) and rarely produced a sound. In a third, separate condition, each button produced a sound in 50% of the trials. Unexpected sounds and unexpected sound omissions elicited a series of error-related brain responses in the electroencephalogram (EEG) at different levels of auditory processing, including a mismatch negativity (MMN) and the P3 complex for unexpected sounds, and the oN1, oN2, and oP3 complex for unexpected omissions. Moreover, unexpected sounds elicited an equivalent MMN-regardless of whether silence was expected (80%) or no reliable expectation was possible (50%), while later P3 components showed different amplitudes. Our results demonstrate flexible action-sound predictions at sensory and higher cortical levels. Furthermore, they indicate that predicted silence does not have an explicit sensory representation at lower levels but emerges at later stages, when higher-level information has been integrated. ### Competing Interest Statement The authors have declared no competing interest.
When a picture is repeatedly named in the context of seman-tically related pictures (homogeneous context), responses areslower than when the picture is repeatedly named in the contextof unrelated pictures (heterogeneous context). This semanticinterference effect in blocked-cyclic naming plays an impor-tant role in devising theories of word production. W & ouml;hner,M & auml;debach, and Jescheniak [W & ouml;hner, S., M & auml;debach, A., &Jescheniak, J. D. Naming pictures and sounds: Stimulus typeaffects semantic context effects.Journal of ExperimentalPsychology: Human Perception and Performance, 47,716-730, 2021] have shown that the effect is substantiallylarger when participants name environmental sounds thanwhen they name pictures. We investigated possible reasonsfor this difference, using EEG and pupillometry. The behavioraldata replicated W & ouml;hner and colleagues. ERPs were more posi-tive in the homogeneous compared with the heterogeneouscontext over central electrode locations between 140-180 msecand 250-350 msec for picture naming and between 250 and350 msec for sound naming, presumably reflecting semanticinterference during semantic and lexical processing. The latercomponent was of similar size for pictures and sounds. ERPswere more negative in the homogeneous compared with theheterogeneous context over frontal electrode locationsbetween 400 and 600 msec only for sounds. The pupillometricdata showed a stronger pupil dilation in the homogeneouscompared with the heterogeneous context only for sounds.The amplitudes of the late ERP negativity and pupil dilation pre-dicted naming latencies for sounds in the homogeneous con-text. The latency of the effects indicates that the difference insemantic interference between picture and sound namingarises at later, presumably postlexical processing stages closerto articulation. We suggest that the processing of the auditorystimuli interferes with phonological response preparation andself-monitoring, leading to enhanced semantic interference.
The ability to focus on relevant information while ignoring distractions is critical during childhood, as it supports learning, social interaction, and adaptation to changing environments. This attentional balance is thought to depend in part on arousal regulation, mediated by the activity of the locus coeruleus norepinephrine (LC-NE) system. Moderate levels of arousal are typically associated with optimal cognitive performance. However, the interaction between arousal and attention remains poorly understood in humans, especially during development. In this study, we investigated whether experimentally modulating tonic arousal, the baseline level of physiological alertness, affects attentional processing in children (N = 44, aged 6–8) and adults (N = 46, aged 18–35). Participants performed an active auditory three-stimulus (standard, novel, target) oddball task, designed to assess selective attention to target tones and distraction by novel sounds. Prior to each task block, tonic arousal was manipulated using music or videos varying in arousing content. Physiological responses were recorded continuously (skin conductance, pupil dilation, heart rate) to index both tonic arousal and transient, phasic changes in arousal triggered by task events. While tonic arousal modulation was successful, as confirmed by skin conductance levels, Bayesian analyses provided evidence for no effect of this modulation on subsequent attentional processing. Importantly, children generally exhibited stronger phasic arousal responses, particularly to task-irrelevant novel sounds, reflecting less mature regulation of attention and arousal. These findings show developmental differences in physiological responses to unexpected environmental stimuli and provide physiological evidence of increased distractibility during childhood. ### Competing Interest Statement The authors have declared no competing interest.
Recent theories describe perception as an inferential process based on internal predictive models that are adjusted by prediction violations (prediction error). Two modulations of the auditory N1 event-related brain potential component have been interpreted as reduced or enhanced prediction error for predictable sensory input: The sound-related N1 component is attenuated for self-generated sounds compared to the N1 elicited by externally generated sounds (N1 suppression). An omission-related component in the N1 time-range is elicited when the self-generated sounds are occasionally omitted (omission N1). We wanted to confirm that both N1 suppression and omission N1 are sensitive to the predictability of sound identity, as reported in the literature. We manipulated the predictability of sound identity in a self-generation paradigm in which button presses in one condition always produced the same sound or in another condition produced a sound randomly selected from a large set of sounds. Omission N1 was modulated by manipulating the predictability of sound identity but surprisingly N1 suppression was not. This contradicts previous reports, challenges prediction-related interpretations of the N1 suppression, and supports alternative explanations for N1 suppression like action-related unspecific sensory gating.
The N2pc is widely employed as an electrophysiological marker of an attention allocation. This interpretation was largely driven by the observation of an N2pc elicited by an isolated relevant target object, which was reported as Experiment 2 in Eimer (1996). All subsequent refined interpretations of the N2pc had to take this crucial finding into account. Despite its central role for neurocognitive attention research, there have been no direct replications and only few conceptual replications of this seminal work. Within the context of #EEGManyLabs, an international community-driven effort to replicate the most influential EEG studies ever published, the present study was selected due to its strong impact on the study of selective attention. We revisit the idea of the N2pc being an indicator of attentional selectivity by delivering a high powered direct replication of Eimer's work through analysis of 779 datasets acquired from 22 labs across 14 countries. Our results robustly replicate the N2pc to form stimuli, but a direct replication of the N2pc to color stimuli technically failed. We believe that this pattern not only sheds further light on the functional significance of the N2pc as an electrophysiological marker of attentional selectivity, but also highlights a methodological problem with selecting analysis windows a priori. By contrast, the consistency of observed ERP patterns across labs and analysis pipelines is stunning, and this consistency is preserved even in datasets that were rejected for (ocular) artifacts, attesting to the robustness of the ERP technique and the feasibility of large-scale multilab EEG (replication) studies.
The sensory input arising from our own movements is predictable to varying degrees. This predictability plays a role in action selection, initiation, and the evaluation of action outcomes, and should therefore influence neural processing both before and after movement. Here, we examined the effect of sensory predictability on pre- and post-movement processing, as reflected in two signals in the human electroencephalogram (EEG): theReadiness Potential(RP) and thepost-movement beta rebound(PMBR). Thirty-six participants performed self-paced button presses in three conditions, in which they received either highly predictable auditory feedback (always the same sound), weakly predictable auditory feedback (an unpredictable, pseudo-random sound), or no auditory feedback. In the former two conditions, 20% of button presses unexpectedly elicited no sound. In these omission trials, we observed brain responses, which indicated that participants formed sensory predictions of a varying level of detail. Crucially, RP amplitude was increased when button presses always produced the same sound, compared to pseudo-random sounds, or no sound, with the latter two conditions showing no difference. In contrast, the PMBR was unchanged across conditions. Our results indicate that the specificity of sensory prediction influences the pre-movement processes reflected in the RP, even when these details are not task-relevant.
Assessments of listening effort are increasingly relevant to understanding the speech-comprehension difficulties experienced by older adults. Pupillometry is the most common tool to assess listening effort but has limitations. Recent research has shown that eye movements decrease when listening is effortful and proposed indicators of eye movements as alternative measures. However, much of the work was conducted in younger adults in trialbased sentence-listening paradigms during concurrent visual stimulation. The extent to which eye movements index listening effort during continuous speech listening, independently of visual stimuli, and in older adults, is unknown. In the current study, younger and older adults listened to continuous stories with varying degrees of background noise under free and moving-dots viewing conditions. Eye movements decreased (as indexed by fixation duration, gaze dispersion, and saccade rate) with increasing speech masking. The reduction in eye movements did not depend on age group or viewing conditions, indicating that eye movements can be used to assess effects of speech masking in different visual situations and in people of different ages. The pupil size was only sensitive to speech masking early in the experiment. In sum, the current study suggests that eye movements are a potential tool to assess listening effort during continuous speech listening.