Predictions concerning upcoming visual input play a key role in resolving percepts. Sometimes input is surprising, under which circumstances the brain must calibrate erroneous predictions so that perception is veridical. Despite the extensive literature investigating the nature of prediction error signalling, it is still unclear how this process interacts with the functionally segregated nature of the visual cortex, particularly within the temporal domain. Here, we recorded electroencephalography (EEG) from humans (N = 32) whilst they viewed static image trajectories containing a bound object that sequentially changed along different visual attribute dimensions (shape and colour). Crucially, the context of this change was designed to appear random (and unsurprising) or violate the established trajectory (and cause a surprise). Event-related potential analysis found no effects of surprise after controlling for cortical adaptation. However, multivariate pattern analyses found whole-scalp neural representations of visual surprise that overlapped between attributes, albeit at distinct, attribute-specific latencies. These findings suggest that visual surprise results in generalised (i.e., attribute-agnostic) prediction error responses that conform to an attribute-dependent temporal hierarchy.
Onset primacy is a behavioural phenomenon whereby humans identify the appearance of an object (onset) with greater efficiency than other kinds of visual change, such as the disappearance of an object (offset). The default mode hypothesis explains this phenomenon by postulating that the attentional system is optimised for onset detection in its initial state. The present study extended this hypothesis by combining a change-detection task and measurement of the P300 event-related potential, which was thought to index the amount of processing resources available to detecting onsets and offsets. In an experiment, while brain activity was monitored by electroencephalography, participants indicated the locations of onsets and offsets under the condition in which they occurred equally often in the same locations across trials. Although there was no reason to prioritise detecting one type of change over the other, onsets were detected more quickly, and they evoked a larger P300 than offsets. These results suggest that processing resources are preferentially allocated to onset detection. This biased allocation may be a basis on which the attentional system defaults to the 'onset detection' mode.
To understand how the human brain distinguishes itself from external stimulation, it was examined if motor predictions enable healthy adult volunteers to infer self-location and to distinguish their body from the environment (and other agents). By uniquely combining a VR-setup with full-body motion capture, a full-body illusion paradigm (FBI) was developed with different levels of motion control: (A) a standard, passive FBI in which they had no motion control; (B) an active FBI in which they made simple, voluntary movements; and (C) an immersive game in which they real-time controlled a human-sized avatar in third person. Systematic comparisons between measures revealed a causal relationship between (i) motion control (prospective agency), (ii) self-other identification, and (iii) the ability to locate oneself. Healthy adults could recognise their movements in a third-person avatar and psychologically align with it (action observation); but did not lose a sense of place (self-location), time (temporal binding), nor who they are (self/other). Instead, motor predictions enabled them to localise their body and to distinguish self from other. In the future, embodied games could target and strengthen the brain's control networks in psychosis and neurodegeneration; real-time motion simulations could help advance neurorehabilitation techniques by fine-tuning and personalising therapeutic settings.
Posttraumatic growth (PTG) is the advantageous change some people report following the struggle to overcome traumatic life circumstances. As neural understanding of PTG is limited, debate persists regarding whether PTG represents "real" or "illusory" change. This study presents a novel supervised machine learning examination, predicting high versus low PTG from electroencephalographic (EEG) data collected from 66 trauma-exposed individuals. Alpha and gamma EEG frequency power accurately classified PTG and demonstrated the disruptive neural influence of posttraumatic stress disorder. Results provide objectively measurable neural evidence of the existence of PTG and the first whole-brain, high-density EEG scalp topographies of PTG in known literature.
Humans use socially relevant stimuli to guide perceptual processing of the surrounding environment, with emotional stimuli receiving preferential attention due to their social importance. Predictive coding theory asserts this cognitive process occurs efficiently by combining predictions about what is to be perceived with incoming sensory information, generating prediction errors that are then used to update future predictions. Recent evidence has identified differing neural activity that demonstrates how spatial and feature-based attention may interact with prediction, yet how emotion-guided attention may influence this relationship remains unknown. In the present study, participants viewed a display of two faces in which attention, prediction, and emotion were manipulated, and responded to a face expressing a specific emotion (anger or happiness). The N170 was found to be enhanced by unpredictable as opposed to predictable stimuli, indicating that it indexes general prediction error signalling processes. The N300 amplitudes were also enhanced by unpredictable stimuli, but they were also affected by the attentional status of angry but not happy faces, suggesting that there are differences in prediction error processes indexed by the N170 and N300. Overall, the findings suggest that the N170 and N300 both index violations of expectation for spatial manipulations of stimuli in accordance with prediction error responding processes.
Predictive coding theories assert that perceptual inference is a hierarchical process of belief updating, wherein the onset of unexpected sensory data causes so-called prediction error responses that calibrate erroneous inferences. Given the functionally specialised organisation of visual cortex, it is assumed that prediction error propagation interacts with the specific visual attribute violating an expectation. We sought to test this within the temporal domain by applying time-resolved decoding methods to electroencephalography (EEG) data evoked by contextual trajectory violations of either brightness, size, or orientation within a bound stimulus. We found that following ∼170 ms post stimulus onset, responses to both size violations and orientation violations were decodable from physically identical control trials in which no attributes were violated. These two violation types were then directly compared, with attribute-specific signalling being decoded from 265 ms. Temporal generalisation suggested that this dissociation was driven by latency shifts in shared expectation signalling between the two conditions. Using a novel temporal bias method, we then found that this shared signalling occurred earlier for size violations than orientation violations. To our knowledge, we are among the first to decode expectation violations in humans using EEG and have demonstrated a temporal dissociation in attribute-specific expectancy violations.
Individual differences in the ability to habitually regulate emotion may impact the efficacy of fear memory extinction. The aim of this study was to assess the relationship between dispositional cognitive reappraisal and expressive suppression with post-retrieval and standard extinction. Fear memory and extinction were measured with the recovery of skin conductance responses. We also examined the relationship between a temporal feature of electrodermal responding (half-recovery time) and each of the emotion regulation strategies. University students (N = 80) underwent a three-day fear conditioning procedure using a within-subject design consisting of acquisition on day one, post-retrieval extinction and standard extinction on day two, and recovery test on day three. Individual difference data on self-reported levels of cognitive reappraisal, expressive suppression, trait anxiety, and depression were collected. We did not detect a relationship between the two emotion regulation strategies measured in this study and acquisition or extinction. We found, however, that increased dispositional use of cognitive reappraisal was associated with lower spontaneous recovery to both the post-retrieval extinction and standard extinction stimulus after controlling for age, trait anxiety, and depression. There were no associations between expressive suppression and conditioned responses. We also observed patterns of faster dissipation of arousal for reappraisal and slower for suppression to the conditioned stimulus during extinction training, which may represent the unique influence of each emotion strategy on the regulation of fear. We conclude greater daily use of cognitive reappraisal, but not expressive suppression, associates with extinction retention after receiving both standard and post-retrieval extinction.
How does the brain distinguish between the signals it produces and the sensations it registers from the environment? To shed light on this, it was investigated if the human mind could be capable of perceiving an avatar’s body in a third person game as one’s own. To create precise, high-quality motion simulations, we uniquely combined a Virtual Reality-setup (Valve Index) with real-time motion capture (Vicon). In doing so, we systematically explored if (predictions of) self-actions allow the brain to infer self-location and to distinguish the body from the environment including other agents. A full-body illusion paradigm (FBI) was developed in VR with three movement conditions: (A) a standard, passive FBI in which people had no motion control; (B) an active FBI in which they made simple voluntary movements; and (C) an immersive game in which they real-time controlled a full-sized human avatar in third person (i.e., the first third person VR-game). Systematic comparisons between measures (implicit, explicit, exit-interview, and temporal binding) revealed a causal relationship between (i) sense of agency, (ii) self-other identification, and (iii) the ability to locate oneself. A loss in sense of agency was reported when movement was restricted, and a shift in self-location and self-identification towards the virtual body was experienced; which did not happen when healthy volunteers were (to some extent) able to voluntarily move. It is confirmed that motor predictions are salient cues for the brain that not only provide a sense of control in self-actions, but also recognition of the self in time and place. People can recognise their movements in a third-person avatar and psychologically align with it (action observation); but do not seem to lose a sense of place (self-location), time (temporal binding), nor who they are (self vs. other), because voluntary action codes the bodily self to a physical location in space. These results provide further evidence for our hypothesis (de Boer et al., 2020) and may shed light on how bodily self-consciousness is constructed. In the future, immersive game simulations could target and strengthen the brain’s control networks in psychosis, neurodegeneration (e.g., dementia, movement disorders) and old age. In addition, real-time motion simulations could help advance future rehabilitation techniques (e.g., to treat nervous system injury) by fine-tuning and personalising the therapeutic setting on demand.
During visual perception, the brain must combine its predictions about what is to be perceived with incoming relevant information. The present study investigated how this process interacts with attention by using event-related potentials that index these cognitive mechanisms. Specifically, this study focused on examining how the amplitudes of the N170, N2pc, and N300 would be modulated by violations of expectations for spatial and featural attributes of visual stimuli. Participants viewed a series of shape stimuli in which a salient shape moved across a set of circular locations so that the trajectory of the shape implied the final position and shape of the stimulus. The final salient stimuli occurred in one of four possible outcomes: predictable position and shape, predictable position but unpredictable shape, unpredictable position but predictable shape, and unpredictable position and shape. The N170 was enhanced by unpredictable positions and shapes, whereas the N300 was enlarged only by unpredictable positions. The N2pc was not modulated by violations of expectations for shapes or positions. Additionally, it was observed post-hoc that the P1pc amplitude was increased by unpredictable shapes. These findings revealed that incorrect prediction increases neural activity. Furthermore, they suggest that prediction and attention interact differently in different stages of visual perception, depending on the type of attention being engaged: The N170 indexes initial prediction error signalling irrespective of the type of information (spatial or featural) in which error occurs, followed by the N300 as a marker of prediction updating involving reorientation of spatial attention.
Identifying the faces of familiar persons requires the ability to assign several different images of a face to a common identity. Previous research showed that the occipito-temporal cortex, including the fusiform and the occipital face areas, is sensitive to personal identity. Still, the viewpoint, facial expression and image-independence of this information are currently under heavy debate. Here we adapted a rapid serial visual stimulation paradigm Johnston et al. (2016, https://doi.org/10.1016/j.cortex.2016.10.002) and presented highly variable ambient-face images of famous persons to measure functional magnetic resonance imaging (fMRI) adaptation. fMRI adaptation is considered as the neuroimaging manifestation of repetition suppression, a neural phenomenon currently explained as a correlate of reduced predictive error responses for expected stimuli. We revisited the question of image-invariant identity-specific encoding mechanisms of the occipito-temporal cortex, using fMRI adaptation with a particular interest in predictive mechanisms. Participants were presented with trials containing eight different images of a famous person, images of eight different famous persons or seven different images of a particular famous person followed by an identity change to violate potential expectation effects about person identity. We found an image-independent adaptation effect of identity for famous faces in the fusiform face area. However, in contrast to previous electrophysiological studies, using similar paradigms, no release of the adaptation effect was observed when identity-specific expectations were violated. Our results support recent multivariate pattern analysis studies, showing image-independent identity encoding in the core face-processing areas of the occipito-temporal cortex. These results are discussed in the frame of recent identity-processing models and predictive mechanisms.
Humans are constantly exposed to a rich tapestry of visual information in a potentially changing environment. To cope with the computational burden this engenders, our perceptual system must use prior context to simultaneously prioritise stimuli of importance and suppress irrelevant surroundings. This study investigated the influence of prediction and attention in visual perception by investigating event-related potentials (ERPs) often associated with these processes, N170 and N2pc for prediction and attention, respectively. A contextual trajectory paradigm was used which violated visual predictions and neglected to predetermine areas of spatial interest, to account for the potentially unpredictable nature of a real-life visual scene. Participants ( N =36) viewed a visual display of cued and non-cued shapes rotating in a five-step predictable trajectory, with the fifth and final position of either the cued or non-cued shape occurring in a predictable or unpredictable spatial location. To investigate the predictive coding theory of attention we used factors of attention and prediction, whereby attention was manipulated as either cued or non-cued conditions, and prediction manipulated in either predictable or unpredictable conditions. Results showed both enhanced N170 and N2pc amplitudes to unpredictable compared to predictable stimuli. Stimulus cueing status also increased N170 amplitude, but this did not interact with stimulus predictability. The N2pc amplitude was not affected by stimulus cueing status. In accordance with previous research these results suggest the N170 is in part a visual prediction error response with respect to higher-level visual processes, and furthermore the N2pc may index attention reorientation. The results demonstrate prior context influences the sensitivity of the N170 and N2pc electrophysiological responses. These findings add further support to the role of N170 as a prediction error signal and suggest that the N2pc may reflect attentional reorientation in response to unpredicted stimulus locations.
The prospective identification of children likely to develop schizophrenia is a vital tool to support early interventions that can mitigate the risk of progression to clinical psychosis. Electroencephalographic (EEG) patterns from brain activity and deep learning techniques are valuable resources in achieving this identification. We propose automated techniques that can process raw EEG waveforms to identify children who may have an increased risk of schizophrenia compared to typically developing children. We also analyse abnormal features that remain during developmental follow-up over a period of ~4 years in children with a vulnerability to schizophrenia initially assessed when aged 9 to 12 years. EEG data from participants were captured during the recording of a passive auditory oddball paradigm. We undertake a holistic study to identify brain abnormalities, first by exploring traditional machine learning algorithms using classification methods applied to hand-engineered features (event-related potential components). Then, we compare the performance of these methods with end-to-end deep learning techniques applied to raw data. We demonstrate via average cross-validation performance measures that recurrent deep convolutional neural networks can outperform traditional machine learning methods for sequence modeling. We illustrate the intuitive salient information of the model with the location of the most relevant attributes of a post-stimulus window. This baseline identification system in the area of mental illness supports the evidence of developmental and disease effects in a pre-prodromal phase of psychosis. These results reinforce the benefits of deep learning to support psychiatric classification and neuroscientific research more broadly.
In the domain of machine learning, Neural Memory Networks (NMNs) have recently achieved impressive results in a variety of application areas including visual question answering, trajectory prediction, object tracking, and language modelling. However, we observe that the attention based knowledge retrieval mechanisms used in current NMNs restrict them from achieving their full potential as the attention process retrieves information based on a set of static connection weights. This is suboptimal in a setting where there are vast differences among samples in the data domain; such as anomaly detection where there is no consistent criteria for what constitutes an anomaly. In this paper, we propose a plastic neural memory access mechanism which exploits both static and dynamic connection weights in the memory read, write and output generation procedures. We demonstrate the effectiveness and flexibility of the proposed memory model in three challenging anomaly detection tasks in the medical domain: abnormal EEG identification, MRI tumour type classification and schizophrenia risk detection in children. In all settings, the proposed approach outperforms the current state-of-the-art. Furthermore, we perform an in-depth analysis demonstrating the utility of neural plasticity for the knowledge retrieval process and provide evidence on how the proposed memory model generates sparse yet informative memory outputs.
European Journal of NeuroscienceVolume 52, Issue 11 p. 4403-4410 EDITORIAL Free Access Editorial: Where the rubber meets the road in visual perception: High temporal-precision brain signals to top-down and bottom-up influences on perceptual resolution Sumie Leung, Queensland University of Technology, Kelvin Grove, Qld, AustraliaSearch for more papers by this authorPatrick Johnston, Corresponding Author patrick.johnston@qut.edu.au orcid.org/0000-0001-7703-1073 Queensland University of Technology, Kelvin Grove, Qld, Australia Correspondence Patrick Johnston, Queensland University of Technology, Victoria Park Road, Kelvin Grove, Qld. 4059, Australia. Email: patrick.johnston@qut.edu.auSearch for more papers by this authorAlan Pegna, School of Psychology, The University of Queensland, Brisbane, Qld, AustraliaSearch for more papers by this authorAina Puce, orcid.org/0000-0002-0716-4185 Department of Psychological and Brain Sciences, Indiana University Bloomington, Bloomington, IN, USASearch for more papers by this authorLisa Scott, orcid.org/0000-0002-8136-891X Department of Psychology, University of Florida, Gainesville, FL, USASearch for more papers by this author Sumie Leung, Queensland University of Technology, Kelvin Grove, Qld, AustraliaSearch for more papers by this authorPatrick Johnston, Corresponding Author patrick.johnston@qut.edu.au orcid.org/0000-0001-7703-1073 Queensland University of Technology, Kelvin Grove, Qld, Australia Correspondence Patrick Johnston, Queensland University of Technology, Victoria Park Road, Kelvin Grove, Qld. 4059, Australia. Email: patrick.johnston@qut.edu.auSearch for more papers by this authorAlan Pegna, School of Psychology, The University of Queensland, Brisbane, Qld, AustraliaSearch for more papers by this authorAina Puce, orcid.org/0000-0002-0716-4185 Department of Psychological and Brain Sciences, Indiana University Bloomington, Bloomington, IN, USASearch for more papers by this authorLisa Scott, orcid.org/0000-0002-8136-891X Department of Psychology, University of Florida, Gainesville, FL, USASearch for more papers by this author First published: 10 November 2020 https://doi.org/10.1111/ejn.15036 Edited by: John Foxe [Correction added on 9 December 2020, after first online publication: Peer review history statement has been added.] AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinked InRedditWechat 1 INTRODUCTION Analysis of human brain responses from various neuroimaging techniques (EEG, S-EEG, ECoG or MEG) to visual stimuli in both the time and the time–frequency domain have provided means for researchers to understand the early stages of visual perception in humans. In the time domain, event-related potentials (ERPs) have been used as neurocognitive tools to study a wide range of cognitive processing in humans, for example, attention, vigilance, arousal, since the 1960s (e.g., Eason et al., 1969; Haider et al., 1964; Spong et al., 1965). Importantly, the early neurophysiological studies also drew attention to the importance of carefully characterizing the visual stimulus—to account for nonlinear behaviour of ERPs especially Steady State Visual Evoked Potentials (van der Tweel & Lunel, 1965; van der Tweel & Spekreijse, 1969). Over the years, evidence from visual ERP components from time-domain analyses reveals a hierarchy in visual processing, with the extraction of stimulus invariance at earlier latencies and increased abstraction levels at later latencies. Specifically, ERP components elicited as early as 100–200 ms post-stimulus often represent categorical sensitivity to stimulus type (Woodman, 2010). For example, the P1 and N1 have been linked to sensory or perceptual processing and the N1 or N170 modulated by expert recognition and visual discrimination (Bentin et al., 1996; Curran et al., 2009; Hillyard et al., 1998; Scott et al., 2006, 2008; Vogel & Luck, 2000). Mid-latency ERP components in the 200–350 ms range often reflect individuation of particular exemplars. For example, the N2 and N250 have been linked to object recognition and categorization (Folstein & Van Petten, 2008; Scott et al., 2006; Scott et al., 2008) and the P3 has been linked to stimulus uncertainty, categorization, context updating and cognitive load (Polich & Herbst, 2000; Sutton et al., 1965). Unlike these other potentials, the mismatch negativity (MMN) appeared to be elicited to infrequent events irrespective of whether the subject paid attention to the stimulus stream or not (Näätänen et al., 1978; Näätänen & Michie, 1979). In this visual processing hierarchy, the extent to which these visual ERP components within different time windows are modulated by top-down versus bottom-up effects is not well understood (e.g., Kinchla & Wolfe, 1979; Melloni et al., 2012; Rauss & Pourtois, 2013). As such, these signals offer the opportunity for exploring the interface between bottom-up and top-down influences on perceptual resolution—where, so to speak, the rubber meets the road. Much of the work in this area has focused on a very particular type of visual stimulus—the human face. However, it remains hotly debated whether the study of such stimuli reveals general properties of the visual system as opposed to the function of highly specialized processing modules that have evolved to support the perception of this critical stimulus class (e.g., Riddoch et al., 2008; McKone and Robbins, 2012). Indeed, there is growing evidence that early to mid-latency (100–350 ms) visual ERP components, including the vMMN, N170 and the N250 may be modulated by non-face objects and visual scenes in interesting and informative ways by factors such as expertise, prediction error and perceptual surprise (Johnston et al., 2017; Stefanics et al., 2014; Tanaka & Curran, 2001). This special issue seeks to bring together a series of empirical papers that will lay out the latest understanding of high temporal resolution evoked brain signals to non-face stimuli, and how they inform our understanding of the mechanisms of visual perception. We first provide an introduction of these papers and showcase them in this order: (1) papers that examined spatial attention; (2) papers that focused on prediction coding; (3) papers on object processing and finally (4) a paper on the processing of body image. Two groups focused on spatial attention, with researchers examining how brain responses were affected by visual stimuli being processed outside the task field (File et al., 2018; Harris et al., 2018). Harris et al. (2018) used peripheral probes in a no-report paradigm to examine awareness of the probes, which was indexed by Visual Awareness Negativity (VAN) and alpha EEG power reduction. VAN is a negativity generated by comparing responses to consciously perceived versus missed stimuli (for review see Koivisto & Revonsuo, 2010). Alpha band activities have been commonly linked to the allocation of spatial attention (Foxe et al., 1998; Foxe & Snyder, 2011; Klimesch, 2012). In the second paper, File and colleagues (2018) examined change detection responses by (a) measuring change and post-change visual mismatch negativity (vMMN) responses and (b) examining how the distance between the task-related and vMMN-related stimuli affected neural responses. In addition to File and colleagues vMMN findings, two other groups focused on prediction error, with one group using spatially related stimuli (Robinson et al., 2018) and the other using object stimuli (Stefanics et al., 2018). Robinson and colleagues (2018) examined N1/N170 to contextually unexpected stimuli that violate rotational trajectories. Stefanics and colleagues (2018) examined repetition suppression and repetition enhancement effects during automatic object processing. Two groups in this special issue examined early visual ERP components (e.g., N1/N170, N250, P2) to object perception/recognition (Jones et al., 2018; Leek et al., 2018). Specifically, Jones et al. (2018) further specified how the N170 and N250 are modulated by image manipulations, as well as basic- and subordinate-level object learning. Leek et al. (2018) examined how contours, surfaces and volumes of three-dimensional (3D) object shape impacted these components. While these papers focused on object processing, Proverbio and colleagues (2018) investigated N1/N170 responses to two formats of numbers (digital vs. verbal), performing source reconstruction to examine differential brain activation associated with these two number formats. The final paper of this group used behavioural ratings to determine which waist-to-hip ratio was the most attractive, and they showcased how the early P1 and N1 components are modulated by attention and waist-to-hip ratio (Del Zotto et al., 2018). 2 VISUAL AWARENESS–SPATIAL ATTENTION Studying the neural correlates of perceptual awareness can allow researchers to infer processes related to consciousness (Crick & Koch, 1990). Yet, different neural correlates of visual awareness can emerge (1) at different temporal stages of processing, for example, visual awareness negativity (VAN) at around 200–400 ms versus P3b which could peak at around 450 ms (Rutiku et al., 2015, Pitts et al., 2012; see review by Railo et al., 2011); and (2) within different brain regions, for example, anterior versus posterior cortical areas (Boly et al., 2017, see review by Hutchinson 2019). Thus, it is not surprising that researchers have developed contrasting views on the degree that top-down versus bottom-up processing contributes to their consciousness models. Dehanene and colleagues' global neuronal workspace theory involves global neuronal workspace and a significant top-down component (Dehanene & Naccache 2001), whereas Lamme and colleagues' recurrent processing model involves primarily bottom-up and horizontal connections (Lamme & Roelfsema, 2000). Inattentional blindness is one method with which awareness can be manipulated. According to a recent review on the neural correlates of inattentional blindness by Hutchinson (2019), the most consistently reported electrophysiological correlates of awareness from inattentional blindness paradigms are the VAN and post-stimulus alpha band suppression. VAN is a negativity occurring between 200 and 400 ms that is localized to a region closed to lateral occipital cortex (Pitts et al., 2012). Lower alpha power in the contralateral parietal region was generated during visual spatial tasks when attention was directed to one visual hemifield (Worden et al., 2000; Yamagishi et al., 2003). The primary aim of Harris and colleagues (2018) was to clarify some ambiguities in the alpha power and awareness link from previous studies. They used a no-report inattentional blindness paradigm to examine these two correlates of awareness—VAN and alpha EEG power changes—to task-irrelevant, non-salient stimuli in the periphery. They found that when participants were aware of the probes, a posterior VAN at around 260–320 ms contralateral to probe location was reported, and this response was not observed when participants did not perceive the probes. Posterior alpha EEG power suppression at around 300–730 ms was produced in the same conditions where VAN was observed. Given that the stimuli were task-irrelevant and non-salient, and participants' lack of knowledge of the probes was reported in the "unaware" condition, Harris and colleagues concluded that the post-stimulus alpha power reduction was associated with awareness itself, but not due to a consequence of attention-related confounds. 3 CHANGE DETECTION AND SPATIAL ATTENTION Mismatch negativity (MMN) has provided invaluable evidence for studying automatic change detection and regularity violations in both auditory and visual processing hierarchies (e.g., Escera et al., 2014; Czigler et al., 2002). Recently, the visual MMN (vMMN) has been reconceptualized as a perceptual prediction error signal, which occurs as the result of discrepancies between the visual stimulus and the brain's probabilistic model of the incoming (predicted) input (Friston, 2005; Stefanics et al., 2014; also see Hari & Puce et al. 2017). MMN is known as for its "automatic" nature. As already mentioned above, it can be generated even if participants are not paying attention to the stimulus sequence. While auditory MMN paradigms tend to use a visual task to direct participants' attention away from the sounds, most visual MMN (vMMN) paradigms use a visual task to direct participants' attention away from the vMMN stimulus sequence. Given that both task and vMMN sequence are in the same modality, it is crucial to understand how the nature of the task (e.g., task difficulty) and the relation between task and vMMN-related stimuli could influence vMMN. While attention-demanding predictive processes (as indicated by task load) could have an impact on vMMN (Kimura & Takeda, 2013), little has been done to study the relation between the task and the vMMN sequence itself. In this special issue, File and colleagues examined how spatial distance affects the focus of attention on a task-irrelevant vMMN sequence using vanishing object parts as deviants. In a separate eye-tracking task, they have also ensured participants' focus of attention remains in the task field. The researchers found that vanishing deviant parts of stimulus objects elicited vMMN at around 150–200 ms, which was not affected by the distance from the task field. The deviants also generated a late posterior positivity at 270–330 ms, which was larger when task field was nearer to the sequence, indicating an orientation initiated by the deviants. Automatic detection of predictable whole object reappearance (as indicated by vMMN generation) was only in the visual field close to the focus of attention. 4 PREDICTIVE ERROR–REPETITION EFFECT While File and colleagues (2018) examined MMN responses to disappearance and reappearance of stimuli, their paradigm did not employ a train of repetition stimuli, hence preventing them from examining the repetition-related effect. Repetition-related effects are neural phenomena that can be measured in single neuron recordings (e.g., see reviews by Malmierca et al., 2015; Nelken & Ulanovsky, 2007), in EEG/MEG (e.g., Recasens et al. 2015; Simpson et al., 2015, see review by Schweinberger & Neumann, 2016) and in fMRI (e.g., Snow et al. 2011; Müller et al., 2013), providing invaluable knowledge in various neuroscience areas (Grill-Spector et al., 2006). Various models have been used to explain the repetition effects (Grill-Spector et al., 2006), one of which is the abovementioned prediction error model (Friston, 2005). In this model, a perceptual learning hierarchical system receives bottom-up input from the level below and top-down predictions from the level above. When stimuli are repeated, prediction error (as indexed by repetition suppression) is suppressed by adjusting the connection strengths between these levels (Baldeweg, 2006). Repetition enhancement tends to occur at a later latency than repetition suppression (Recasens et al. 2015), hence repetition enhancement is believed to represent the process of sharpening perceptual predictions at higher levels in the hierarchy (Auksztulewicz & Friston, 2016). While most repetition-EEG research has focused on face processing (see review by Schweinberger & Neumann, 2016), little has been done to examine the time course and topographic distribution of repetition effects for objects. In this issue, Stefanics and colleagues (2018) examined the repetition effect to unattended objects. Specifically, they used a mass-univariate approach in statistical parametric mapping (SPM) to analyse ERP amplitudes of all sensors at all time points (50–500 ms post-stimulus). They performed Bayesian model comparisons and found that the exponential model was the best model that explained the time course of ERP decay to repeated object stimuli. Their model revealed: (1) repetition suppression effects in three time windows (86–140, 322–360 and 400–446 ms) and with different occipital, temporoparietal and frontotemporal distributions, respectively and (2) repetition enhancement effects at 320–340 ms over the occipitotemporal region. Stefanics and colleagues suggested that these repetition suppression and enhancement effects over time in different brain regions might reflect different stages of perceptual inference occurring at several levels of the cortical hierarchy. 5 PREDICTIVE ERROR–CONTEXT TRAJECTORY In addition to prediction and prediction error, perceptual surprise is another important element in predictive coding models (Friston & Keibel, 2009). While automatic change detection and repetition paradigms providing electrophysiological evidence for predictive coding models, most of these paradigms are not designed to manipulate the level of perceptual surprise. A coherent stimulus trajectory allows accumulation of prior evidence based on the ongoing context, and it also allows researchers to study whether the level of perceptual surprise is modulated by the degree to which the prediction model is violated. To manipulate perceptual surprise in ongoing contexts, a recent study used different types of visual trajectories: facial expression trajectories, body rotation trajectories and location trajectories in four separate experiments (Johnston et al., 2017). In each sequence, the first few images would create a context of implied movement, and the final image would either conform or violate the prevailing contextual trajectory by moving in the opposite direction. N/M170 was strongly modulated by violations of expectations across different contextual trajectories and stimulus types. This led the researchers to posit that N/M170 can index visual processes related to perceptual prediction and error-checking/resolution. Building upon this idea, in the current issue, Robinson and colleagues (2018) examine the issue of whether the N170 prediction error signal is modulated in a dose-dependent manner by the degree of perceptual surprise. To achieve this, they extended Johnston's paradigm in two separate experiments by manipulating (1) the number of images preceding the final transition and (2) the magnitude of trajectory violation in the final transition. As sequence length increased, a smaller N1/N170 response occurred in the predictable sequence and a larger N1/N170 was generated to violation of expectancy in unpredictable sequences. This suggested that more data in the same trajectory strengthens expectancy: when expectancy is confirmed, prediction error is reduced due to reduced surprise; when it is violated, prediction error is increased due to increased surprise. Furthermore, prediction error was largest when the extent of violation was the largest. The conclusion was that the N1/N170 response can serve as a marker of prediction error and its amplitude is dose-dependent on the magnitude of perceptual surprise. 6 OBJECT PROCESSING A range of theoretical models have been proposed to better understand the underlying neural mechanisms in visual object recognition. While some theories involved mostly bottom-up, feedforward networks (e.g., Serre et al. 2007; Kheradpisheh et al., 2016), others stress the importance of top-down facilitation of visuocortical regions (e.g., Bar, 2003; Fenske et al., 2006). Taking advantage of the high temporal resolution property in MEG and EEG, researchers attempted to answer some of the questions related to this debate. For example, Ahlfors et al. (2015) analysed MEG source current directions in a visual object recognition experiment. They argued that the directions of source currents were opposite for expected feedforward and feedback inputs, as represented by early and late visual evoked responses. In the current issue, two papers relating to object recognition advance work in this area with respect to two specific issues: (1) complex 3D object recognition and (2) learned expertise to artificial objects. Leek and colleagues (2018) showed that N1 (140–200 ms), P2 (220–260 ms) and N2 (260–320 ms) components recorded over occipitoparietal electrodes were modulated by contours, surfaces and volumes of complex 3D objects. These results indicate that early perceptual processing, as denoted by N1 (which peaked at around 170 ms) is sensitive to higher-order shape structure of complex 3D objects. In addition, theta band activity was modulated by part feature type during the N1 latency interval. This finding argues against some of the theoretical models of object recognition that propose mostly feedforward networks and do not attribute functional significance to higher-order 3D structure during visual perception (e.g., Serre et al. 2007; Kheradpisheh et al., 2016). A second paper by Jones and colleagues (2018) investigated ERP modulation by learned expertise to artificial objects. The N170 component has previously been shown to be modulated by expertise with non-face images, such as dogs, birds, cars; fingerprints (e.g., Busey & Vanderkolk, 2005; Scott et al., 2006, 2008; Tanaka & Curran, 2001). However, in the context of acquiring perceptual expertise through training, the influence of visual features, such as colour and spatial frequency were previously unknown. Jones and colleagues first examined how subordinate- versus basic-level training of previously unknown computer-generated artificial objects impacted ERP indices of object and face processing. Second, they examined the extent to which image manipulations of colour and spatial frequency disrupt these ERP indices. Both N170 and N250 components, recorded over occipital and occipitotemporal regions, showed a greater increase after subordinate training than after basic-level training. The authors also reported a training-related brain-behaviour correlation between N170 latency and reaction time. Finally, image manipulations did modulate behaviour and the N170 and N250 components, but these effects were independent of expertise. This shows that subordinate-level expert access to non-face stimuli could occur within the first 150–350 ms of object processing. In the context of the visual processing hierarchy, we could argue that these early ERP components are impacted by top-down processes, in addition to bottom-up processes, as expertise-related activity tend to involve top-down engagement (e.g., Harel et al., 2010). 7 NUMBER PROCESSING The occipitotemporal N1/N170 component can also reflect early visual processing of numbers, and can be generated by both number notations: words (verbal) and digits (Arabic) (Dehaene, 1996; Pinel et al., 2001). Different brain regions are responsible for generating the N1/N170 component to the two number notations. Some studies showed that N1/N170 to verbal numbers was larger in the left hemisphere (Dehaene, 1996; Pinel et al., 2001), whereas activation to Arabic numbers was more bilateral (Dehaene, 1996). In contrast, source modelling has found a right hemispheric lateralization for N170 to Arabic numbers (Pinel et al., 2001). Furthermore, a recent MEG study has reported a hemispheric double dissociation for activation at 120–130 ms between verbal and Arabic number processing (Carreiras et al., 2015). In terms of timing, the latency of the N1/N170 component has been reported to be sensitive to number notation: with the verbal-N1 peaking earlier than the Arabic-N1 (Dehaene, 1996). That said, other research did not observe a latency difference with the two number notations (Pinel et al., 2001). Given the inconsistent findings on the role of brain regions and timing of the N1/N170 component, in verbal and Arabic number processing, this area of research still requires clarification. In this special issue, Proverbio and colleagues (2018) attempted to clarify these topics by examining behavioural, ERP responses and source localized activity to numbers presented in Arabic or verbal formats. Faster and more accurate responses were found for digit targets than for verbal numbers. The occipitotemporal N1 component (both peaking at around 180 ms) was generated only in the left hemisphere during verbal number processing, but was elicited bilaterally during digit processing. Source reconstruction over the N1 latency range revealed higher activation to digits than to their verbal counterparts. The left fusiform gyrus (FG, BA37) was the only area activated by both digit and verbal numbers. Verbal numbers have also generated a larger anterior negativity in the 300–500 ms window than Arabic numbers. These data suggested distinct mechanisms underlie number reading through digits versus words, with digits being accessed earlier and more accurately than words. 8 BODY PROCESSING In addition to indexing object and number processing, the occipitotemporal N1/N170 component (150–230 ms) can also represent the visual processing of bodies as a category (van Heijnsbergen et al., 2007; Meeren et al., 2005; Minnebusch & Daum, 2009; Stekelenburg & Gelder, 2004; Thierry et al., 2006). This N170 was unaffected by preferential attention, but was influenced by affective arousal, when stimuli were nude bodies (Hietanen et al., 2014). While N170 was also unaffected by body expression, such as fear, a vertex positive potential (VPP) at a similar time window as N170 tended to be generated by fearful body expressions (Van Heijnsbergen et al., 2007; Stekelenburg & De Gelder, 2004). In this special issue, Del Zotto et al. (2018) examined the perceived attractiveness of a secondary sexual characteristic of the human body—waist-to-hip ratio, in a behavioural task and in an EEG paradigm. They also aimed to test the role of attention and top-down mechanisms by asking the participants to selectively attend to the body stimuli versus not in the ERP task. The attractiveness ratings of the bodies revealed a preference of waist-to-hip ratio of 0.7. In terms of ERP data, the posterior P1 and N1 components (in the ranges of 80–120 ms and 130–170 ms, respectively) were the earliest ERPs to be modulated by attention and bodies. The VPP (in the 120–180 ms range) was more enhanced for stimuli with waist-to-hip ratios of 0.7 compared to other ratios, only in the attend condition, illustrating once again that learned preferences influence early responses to visual stimuli. 9 CONCLUSION Taken together, the set of papers compiled in this Special Issue demonstrate the extent to which visual evoked signals, recorded over posterior scalp regions, reflect the combined influences of both top-down and bottom-up processes in percept resolution. They support growing evidence that early to mid-latency (100–350 ms) visually evoked brain signals to non-face objects (bodies, novel objects, shapes, words and numbers) may be modulated in interesting and informative ways by factors such as increasing expertise, salience and perceptual surprise. Several papers focus on mid-latency components (200–400 ms), which seem likely to index attentional allocation or reorientation and later latency components (400 ms onwards), reflecting elaborated or attentionally mediated processing of salient or unexpected stimuli. In some cases, later latency effects may reflect recalibration of contextual representations. As Stefanics et al. (2018) point out, some of these temporal effects in different brain regions might reflect different stages of the perceptual inference that occurs at several levels of the cortical hierarchy. A number of the papers focus on the early latency N1/N170 component that has been traditionally viewed as having a special sensitivity to faces. The papers in this special issue support a much broader interpretation of this component, as reflecting the collision of incoming sensory information (about many types of visual stimuli) with stored representations that attempt to constrain and explain the incoming signals. According to the papers in this special issue, these representations can be the result of learned expertise, or reflect the current state of an internal model of the current context. It seems likely that these processes are distributed across different parts of occipitotemporal and occipitoparietal cortices, with different areas sustaining representations of different stimuli or stimulus attributes. In general, modulation of these early latency signals by liminality, expertise, expectation or surprise reflects earliest mechanisms of "higher-level" vision, whereby stimuli are flagged for prioritized processing and potentially attentional awareness. PEER REVIEW The peer review history for this article is available at https://publons.com/publon/10.1111/ejn.15036 Open Research PEER REVIEW The peer review history for this article is available at https://publons.com/publon/10.1111/ejn.15036 REFERENCES Ahlfors, S. P., Jones, S. R., Ahveninen, J., Hämäläinen, M. S., Belliveau, J. W., & Bar, M. (2015). Direction of magnetoencephalography sources associated with feedback and feedforward contributions in a visual object recognition task. Neuroscience Letters, 585, 149– 154. https://doi.org/10.1016/j.neulet.2014.11.029 Crossref CAS PubMed Web of Science®Google Scholar Auksztulewicz, R., & Friston, K. (2016). Repetition suppression and its contextual determinants in pred
Recent theories suggest that self-consciousness, in its most elementary form, is functionally disconnected from the phenomenal body. Patients with psychosis frequently misattribute their thoughts and actions to external sources; and in certain out-of-body experiences, lucid states, and dreams body-ownership is absent but self-identification is preserved. To explain these unusual experiences, we hypothesized that self-identification depends on inferring self-location at the right angular gyrus (i.e., perspective-taking). This process relates to the discrimination of self-produced signals (endogenous attention) from environmental stimulation (exogenous attention). Therefore, when this mechanism fails, this causes altered sensations and perceptions. We combined a Full-body Illusion paradigm with brain stimulation (HD-tDCS) and found a clear causal association between right angular gyrus activation and alterations in self-location (perspective-taking). Anodal versus sham HD-tDCS resulted in: a more profound out-of-body shift (with reduced sense of agency); and a weakened ability to discriminate self from other perspectives. We conclude that self-identification is mediated in the brain by inferring self-location (i.e., perspective-taking). Self-identification can be decoupled from the bodily self, explaining phenomena associated with disembodiment. These findings present novel insights into the relationship between mind and body, and may offer important future directions for treating psychosis symptoms and rehabilitation programs to aid in the recovery from a nervous system injury. The brain’s ability to locate itself might be the key mechanism for self-identification and distinguishing self from other signals (i.e., perspective-taking).
Face inversion effects occur for both behavioral and electrophysiological responses when people view faces. In EEG, inverted faces are often reported to evoke an enhanced amplitude and delayed latency of the N170 ERP. This response has been attributed to the indexing of specialized face processing mechanisms within the brain. However, inspection of the literature revealed that, although N170 is consistently delayed to a variety of face representations, only photographed faces invoke enhanced N170 amplitudes upon inversion. This suggests that the increased N170 amplitudes to inverted faces may have other origins than the inversion of the face's structure. We hypothesize that the unique N170 amplitude response to inverted photographed faces stems from multiple expectation violations, over and above structural inversion. For instance, rotating an image of a face upside-down not only violates the expectation that faces appear upright but also lifelong priors about illumination and gravity. We recorded EEG while participants viewed face stimuli (upright vs. inverted), where the faces were illuminated from above versus below, and where the models were photographed upright versus hanging upside-down. The N170 amplitudes were found to be modulated by a complex interaction between orientation, lighting, and gravity factors, with the amplitudes largest when faces consistently violated all three expectations. These results confirm our hypothesis that face inversion effects on N170 amplitudes are driven by a violation of the viewer's expectations across several parameters that characterize faces, rather than a disruption in the configurational disposition of its features.
Research has shown that a single presentation of the conditioned stimulus prior to extinction training can diminish conditioned responses. However, replication has proven difficult and appears to be limited by boundary conditions. Here we tested the boundary condition of memory strength by comparing the effect of reinforcement rate to assess its role in post-retrieval extinction. Eighty university students had undergone a three-day fear conditioning experiment in which two partial reinforcement schedules (40%, 80%) were applied. The findings indicated that both low and high partial reinforcement groups did not demonstrate recovery of conditioned responses after post-retrieval extinction. In contrast, both groups demonstrated significant recovery to standard extinction with significantly greater recovery in the 80% group relative to the 40% group. Additionally, we found that greater physiological arousal during memory retrieval significantly predicted recovery of fear at test phase. We conclude that when compared to a lower partial reinforcement schedule, a higher partial reinforcement resulted in the formation of a stronger memory as indicated by greater physiological arousal during memory reactivation and recovery of conditioned responses after standard extinction, but that it does not function as a boundary condition of post-retrieval extinction. These data are significant because it is the first study to investigate the effect of varying partial reinforcement schedules on fear recovery and add to the body of literature that continue to identify sources of failure in the application of post-retrieval extinction.
Recent theories suggest that self-consciousness, in its most elementary form, is separate from the own body. Patients with psychosis frequently misattribute their thoughts and actions to external sources; and in certain out-of-body experiences, lucid states, and dreams body-ownership is absent but self-identification is preserved. We hypothesized that self-identification depends on inferring self-location at the right Angular Gyrus (perspective-taking). This process relates to the discrimination of self-produced signals (endogenous attention) from environmental stimulation (exogenous attention). We combined a Full-body Illusion paradigm with brain stimulation (HD-tDCS) and found a clear causal association between right Angular Gyrus activation and alterations in self-location (perspective-taking). Anodal versus sham HD-tDCS resulted in: a more profound out-of-body shift (with reduced sense-of-agency); and a weakened ability to discriminate self from other perspectives. We conclude that self-identification is mediated in the brain by inferring self-location (perspective-taking). Self-identification can be decoupled from the bodily self, explaining phenomena associated with disembodiment.
Predictive coding theories of perception highlight the importance of constantly updated internal models of the world to predict future sensory inputs. Importantly, such theories suggest that prediction-error signalling should be specific to the violation of predictions concerning distinct attributes of the same stimulus. To interrogate this as yet untested prediction, we focused on two different aspects of face perception (identity and orientation) and investigated whether cortical regions which process particular stimulus attributes also signal prediction violations with respect to those same stimulus attributes. We employed a paradigm using sequential trajectories of images to create perceptual expectations about face orientation and identity, and then parametrically violated each attribute. Using MEG data, we identified double dissociations of expectancy violations in the dorsal and ventral visual streams, such that the right fusiform gyrus showed greater prediction-error signals to identity violations than to orientation violations, whereas the left angular gyrus showed the converse pattern of results. Our results suggest that perceptual prediction-error signalling is directly linked to regions associated with the processing of different stimulus properties.