Although recent work has made headway in understanding the neural temporospatial dynamics of conscious perception, much of that work has focused on visual paradigms. To determine whether there are shared mechanisms for perceptual consciousness across sensory modalities, here we test within the auditory domain. Participants completed an auditory threshold task while undergoing intracranial electroencephalography. Recordings from >2,800 grey matter electrodes were analyzed for broadband gamma power (a range which reflects local neural activity). For perceived trials, we find nearly simultaneous activity in early auditory regions, the right caudal middle frontal gyrus, and the non-auditory thalamus; followed by a wave of activity that sweeps through auditory association regions into parietal and frontal cortices. For not perceived trials, significant activity is restricted to early auditory regions. These findings show the cortical and subcortical networks involved in auditory perception are similar to those observed with vision, suggesting shared mechanisms for conscious perception.

Research in functional neuroimaging has suggested that category-selective regions of visual cortex, including the ventral temporal cortex (VTC), can be reactivated endogenously through imagery and recall. Face representation in the monkey face-patch system has been well studied and is an attractive domain in which to explore these processes in humans. The VTCs of 8 human subjects (4 female) undergoing invasive monitoring for epilepsy surgery were implanted with microelectrodes. Most (26 of 33) category-selective units showed specificity for face stimuli. Different face exemplars evoked consistent and discriminable responses in the population of units sampled. During free recall, face-selective units preferentially reactivated in the absence of visual stimulation during a 2 s window preceding face recall events. Furthermore, we show that in at least 1 subject, the identity of the recalled face could be predicted by comparing activity preceding recall events to activity evoked by visual stimulation. We show that face-selective units in the human VTC are reactivated endogenously, and present initial evidence that consistent representations of individual face exemplars are specifically reactivated in this manner.SIGNIFICANCE STATEMENT The role of "top-down" endogenous reactivation of native representations in higher sensory areas is poorly understood in humans. We conducted the first detailed single-unit survey of ventral temporal cortex (VTC) in human subjects, showing that, similarly to nonhuman primates, humans encode different faces using different rate codes. Then, we demonstrated that, when subjects recalled and imagined a given face, VTC neurons reactivated with the same rate codes as when subjects initially viewed that face. This suggests that the VTC units not only carry durable representations of faces, but that those representations can be endogenously reactivated via "top-down" mechanisms.

Variations in body habitus (e.g. obesity) have been linked to widespread changes in the brain. However, the whole-brain functional networks associated with individual variations in body habitus have not yet been fully characterized. To investigate this question, we used connectome-based predictive modeling (CPM) to predict body habitus in individual subjects, approximated using body mass index (BMI).

The discovery that deep convolutional neural networks (DCNNs) achieve human performance in realistic tasks offers fresh opportunities for linking neuronal tuning properties to such tasks. Here we show that the face-space geometry, revealed through pair-wise activation similarities of face-selective neuronal groups recorded intracranially in 33 patients, significantly matches that of a DCNN having human-level face recognition capabilities. This convergent evolution of pattern similarities across biological and artificial networks highlights the significance of face-space geometry in face perception. Furthermore, the nature of the neuronal to DCNN match suggests a role of human face areas in pictorial aspects of face perception. First, the match was confined to intermediate DCNN layers. Second, presenting identity-preserving image manipulations to the DCNN abolished its correlation to neuronal responses. Finally, DCNN units matching human neuronal group tuning displayed view-point selective receptive fields. Our results demonstrate the importance of face-space geometry in the pictorial aspects of human face perception.

The unique profile of strong and weak cognitive traits characterizing each individual is of a fundamental significance, yet their neurophysiological underpinnings remain elusive. Here, we present intracranial electroencephalogram (iEEG) measurements in humans pointing to resting-state cortical "noise" as a possible neurophysiological trait that limits visual recognition capacity. We show that amplitudes of slow (<1 Hz) spontaneous fluctuations in high-frequency power measured during rest were predictive of the patients' performance in a visual recognition 1-back task (26 patients, total of 1,389 bipolar contacts pairs). Importantly, the effect was selective only to task-related cortical sites. The prediction was significant even across long (mean distance 4.6 ± 2.8 days) lags. These findings highlight the level of the individuals' internal "noise" as a trait that limits performance in externally oriented demanding tasks.

Functional connectivity derived from functional magnetic resonance imaging data has been extensively used to characterize individual and group differences. While these connectomes have traditionally been constructed using resting-state data, recent work has highlighted the importance of combining multiple task connectomes, particularly for identifying individual differences. Yet, these methods have not yet been extended to investigate differences at the group level. Here, we propose a mass multivariate edge-wise approach to improve the detection of group differences by combining connectomes from multiple sources. For each edge, the magnitude of connection strength from each of multiple connectomes are included in statistical hypothesis testing. We evaluate the proposed approach by estimating sex differences in two large, publicly available datasets: the Human Connectome Project and Philadelphia Neurodevelopmental Cohort. Results indicate the proposed mass multivariate edge-wise analysis offers improved detection of group differences compared to univariate analysis, and support the utility of combining multiple connectomes to improve detection of group differences.

ObjectiveDisconnection of the cerebral hemispheres by corpus callosotomy (CC) is an established means to palliate refractory generalized epilepsy. Laser interstitial thermal therapy (LITT) is gaining acceptance as a minimally invasive approach to treating epilepsy, but this method has not been evaluated in clinical series using established methodologies to assess connectivity. The goal in this study was to demonstrate the safety and feasibility of MRI-guided LITT for CC and to assess disconnection by using electrophysiology- and imaging-based methods.MethodsRetrospective chart and imaging review was performed in 5 patients undergoing LITT callosotomy at a single center. Diffusion tensor imaging and resting functional MRI were performed in all patients to assess anatomical and functional connectivity. In 3 patients undergoing simultaneous intracranial electroencephalography monitoring, corticocortical evoked potentials and resting electrocorticography were used to assess electrophysiological correlates.ResultsAll patients had generalized or multifocal seizure onsets. Three patients with preoperative evidence for possible lateralization underwent stereoelectroencephalography depth electrode implantation during the perioperative period. LITT ablation of the anterior corpus callosum was completed in a single procedure in 4 patients. One complication involving misplaced devices required a second procedure. Adequacy of the anterior callosotomy was confirmed using contrast-enhanced MRI and diffusion tensor imaging. Resting functional MRI, corticocortical evoked potentials, and resting electrocorticography demonstrated functional disconnection of the hemispheres. Postcallosotomy monitoring revealed lateralization of the seizures in all 3 patients with preoperatively suspected occult lateralization. Four of 5 patients experienced > 80% reduction in generalized seizure frequency. Two patients undergoing subsequent focal resection are free of clinical seizures at 2 years. One patient developed a 9-mm intraparenchymal hematoma at the site of entry and continued to have seizures after the procedure.ConclusionsMRI-guided LITT provides an effective minimally invasive alternative method for CC in the treatment of seizures associated with drop attacks, bilaterally synchronous onset, and rapid secondary generalization. The disconnection is confirmed using anatomical and functional neuroimaging and electrophysiological measures.

Hippocampal sharp-wave ripples (SWRs) constitute one of the most synchronized activation events in the brain and play a critical role in offline memory consolidation. Yet their cognitive content and function during awake, conscious behavior remains unclear. We directly examined this question using intracranial recordings in human patients engaged in episodic free recall of previously viewed photographs. Our results reveal a content-selective increase in hippocampal ripple rate emerging 1 to 2 seconds prior to recall events. During recollection, high-order visual areas showed pronounced SWR-coupled reemergence of activation patterns associated with recalled content. Finally, the SWR rate during encoding predicted subsequent free-recall performance. These results point to a role for hippocampal SWRs in triggering spontaneous recollections and orchestrating the reinstatement of cortical representations during free episodic memory retrieval.

Despite the massive accumulation of systems neuroscience findings, their functional meaning remains tentative, largely due to the absence of realistically performing models. The discovery that deep convolutional networks achieve human performance in realistic tasks offers fresh opportunities for such modeling. Here we show that the face-space topography of face-selective columns recorded intra-cranially in 32 patients significantly matches that of a DCNN having human-level face recognition capabilities. Three modeling aspects converge in pointing to a role of human face areas in pictorial rather than person identification: First, the match was confined to intermediate layers of the DCNN. Second, identity preserving image manipulations abolished the brain to DCNN correlation. Third, DCNN neurons matching face-column tuning displayed view-point selective receptive fields. Our results point to a “convergent evolution” of pattern similarities in biological and artificial face perception. They demonstrate DCNNs as a powerful modeling approach for deciphering the function of human cortical networks.

Transcranial electric stimulation (TES) is an increasingly popular method for non-invasive modulation of brain activity and a potential treatment for neuropsychiatric disorders. However, there are concerns about the reliability of its application because of variability in TES-induced intracranial electric fields across individuals. While realistic computational models offer can help to alleviate these concerns, their direct empirical validation is sparse, and their practical implications are not always clear. In this study, we combine direct intracranial measurements of electric fields generated by TES in surgical epilepsy patients with computational modeling. First, we directly validate the computational models and identify key parameters needed for accurate model predictions. Second, we derive practical guidelines for a reliable application of TES in terms of the precision of electrode placement needed to achieve a desired electric field distribution. Based on our results, we recommend electrode placement accuracy to be < 1 cm for a reliable application of TES across sessions.

Whereas the neurophysiology of respiration has traditionally focused on automatic brain stem processes, higher brain mechanisms underlying the cognitive aspects of breathing are gaining increasing interest. Therapeutic techniques have used conscious control and awareness of breathing for millennia with little understanding of the mechanisms underlying their efficacy. Using direct intracranial recordings in humans, we correlated cortical and limbic neuronal activity as measured by the intracranial electroencephalogram (iEEG) with the breathing cycle. We show this to be the direct result of neuronal activity, as demonstrated by both the specificity of the finding to the cortical gray matter and the tracking of breath by the gamma-band (40-150 Hz) envelope in these structures. We extend prior observations by showing the iEEG signal to track the breathing cycle across a widespread network of cortical and limbic structures. We further demonstrate a sensitivity of this tracking to cognitive factors by using tasks adapted from cognitive behavioral therapy and meditative practice. Specifically, volitional control and awareness of breathing engage distinct but overlapping brain circuits. During volitionally paced breathing, iEEG-breath coherence increases in a frontotemporal-insular network, and during attention to breathing, we demonstrate increased coherence in the anterior cingulate, premotor, insular, and hippocampal cortices. Our findings suggest that breathing can act as an organizing hierarchical principle for neuronal oscillations throughout the brain and detail mechanisms of how cognitive factors impact otherwise automatic neuronal processes during interoceptive attention. NEW & NOTEWORTHY Whereas the link between breathing and brain activity has a long history of application to therapy, its neurophysiology remains unexplored. Using intracranial recordings in humans, we show neuronal activity to track the breathing cycle throughout widespread cortical/limbic sites. Volitional pacing of the breath engages frontotemporal-insular cortices, whereas attention to automatic breathing modulates the cingulate cortex. Our findings imply a fundamental role of breathing-related oscillations in driving neuronal activity and provide insight into the neuronal mechanisms of interoceptive attention.

Asked to freely recall items from a predefined set (e.g., animals), we rarely recall a wrong exemplar (e.g., a vegetable). This capability is so powerful and effortless that it is essentially taken for granted, yet, surprisingly, the underlying neuronal mechanisms are unknown. Here we investigate this boundary setting mechanism using intracranial recordings (ECoG), in 12 patients undergoing epilepsy monitoring engaged in episodic free recall. After viewing vivid photographs from two categories (famous faces and places), patients were asked to freely recall these items, targeting each category in separate blocks. Our results reveal a rapid and sustained rise in neuronal activity ("baseline shift") in high-order visual areas that persists throughout the free recall period and reflects the targeted category. We further show a more transient reactivation linked to individual recall events. The results point to baseline shift as a flexible top-down mechanism that biases spontaneous recall to remain within the required categorical boundaries.

A key hallmark of visual perceptual awareness is robustness to instabilities arising from unnoticeable eye and eyelid movements. In previous human intracranial (iEEG) work (Golan et al., 2016) we found that excitatory broadband high-frequency activity transients, driven by eye blinks, are suppressed in higher-level but not early visual cortex. Here, we utilized the broad anatomical coverage of iEEG recordings in 12 eye-tracked neurosurgical patients to test whether a similar stabilizing mechanism operates following small saccades. We compared saccades (1.3°-3.7°) initiated during inspection of large individual visual objects with similarly-sized external stimulus displacements. Early visual cortex sites responded with positive transients to both conditions. In contrast, in both dorsal and ventral higher-level sites the response to saccades (but not to external displacements) was suppressed. These findings indicate that early visual cortex is highly unstable compared to higher-level visual regions which apparently constitute the main target of stabilizing extra-retinal oculomotor influences.

Transcranial electric stimulation (TES) is an emerging technique to non-invasively modulate brain function. However, the spatiotemporal distribution of electric fields during TES remains poorly understood. In this study we perform direct intracranial measurements of the electric field generated by transcranial alternating current (tACS) in epilepsy patients and cebus monkeys and evaluate the capacity of finite element method (FEM) models to predict the spatial distribution of measured electric fields. Two presurgical epilepsy patients, with ca. 100 intracranially implanted electrodes participated in a single TES session. Two sponge electrodes (25 cm2) were attached over the left and right temporal cortex and a current of 1 mA with a frequency of 1 Hz was applied for 2 min. In two cebus monkeys three electrodes, with a total of 32 contacts were permanently implanted with posterior-anterior orientation. In multiple sessions intracranial EEG was recorded during TES. We varied the frequency of stimulation from 1–150 Hz and computed amplitude and phase relationships of recorded voltages. We constructed FEM models with increasing anatomical complexity for one epilepsy patient and compared the measured and simulated electric fields. Voltage magnitude slightly decreased with stimulation frequency up to 10% and small phase differences between electrode contacts up to a few degrees were observed. Electric field strengths were strongest in superficial brain regions with maximum values of 0.5 mV/mm (Download : Download high-res image (1MB)Download : Download full-size imageFig. 1). Comparison of measured and simulated potentials and electric fields showed very high correlation values for the potentials (r = 0.95) and correlations of r = 0.7 for the electric fields. Evaluating the predictive value of increasingly complex FEM models highlighted the importance of accurate skull modeling especially in the vicinity of skull defects (Download : Download high-res image (746KB)Download : Download full-size imageFig. 2). We conducted a comprehensive evaluation of intracranial electric field during TES in both human patients and monkeys. Our results indicate that TES currents spread in a linear ohmic manner and capacitive effects are small indicating that the quasi-static approximation is well justified in the low frequency range. The spatial variation of the electric fields can be captured using realistic FEM models.

Transcranial electric stimulation (TES) is an increasingly popular method to non-invasively modulate brain function. Recently, we have directly measured the electric field distribution in humans and non-human primates. However, in order to derive practical guidelines it is necessary to identify key factors that determine the electric field during TES in a given individual. Here, based on combined measurements and computational modeling, we identify determinant factors to be accounted for for a reliable application of TES.

Transcranial electric stimulation (TES) is an emerging technique, developed to non-invasively modulate brain function. However, the spatiotemporal distribution of the intracranial electric fields induced by TES remains poorly understood. In particular, it is unclear how much current actually reaches the brain, and how it distributes across the brain. Lack of this basic information precludes a firm mechanistic understanding of TES effects. In this study we directly measure the spatial and temporal characteristics of the electric field generated by TES using stereotactic EEG (s-EEG) electrode arrays implanted in cebus monkeys and surgical epilepsy patients. We found a small frequency dependent decrease (10%) in magnitudes of TES induced potentials and negligible phase shifts over space. Electric field strengths were strongest in superficial brain regions with maximum values of about 0.5 mV/mm. Our results provide crucial information of the underlying biophysics in TES applications in humans and the optimization and design of TES stimulation protocols. In addition, our findings have broad implications concerning electric field propagation in non-invasive recording techniques such as EEG/MEG.

An inherent limitation of human visual system research stems from its reliance on highly controlled laboratory conditions. Visual processing in the real world differs substantially from such controlled conditions. In particular, during natural vision, we continuously sample the dynamic environment by variable eye movements that lead to inherent instability of the optical image. The neuronal mechanism by which human perception remains stable under these natural conditions remains unknown. Here, we examined a neural mechanism that may contribute to such stability, i.e., the extent to which neuronal responses remain invariant to oculomotor parameters and viewing conditions. To this end, we introduce an experimental paradigm in which intracranial brain activity, a video of the real-life visual scene, and free oculomotor behavior were simultaneously recorded in human patients. Our results reveal, in high-order visual areas, a remarkable level of neural invariance to the length of eye fixations and lack of evidence for a saccade-related neuronal signature. Thus, neuronal responses, while showing high selectivity to the category of visual images, manifested stable "iconic" dynamics. This property of invariance to fixation onset and duration emerged only in high-order visual representations. In early visual cortex, the fixation onset was accompanied with suppressive neural signal, and duration of neuronal responses was largely determined by the fixation times. These results uncover unique neuronal dynamics in high-order ventral stream visual areas that could play an important role in achieving perceptual stability, despite the drastic changes introduced by oculomotor behavior in real life.

Article Figures and data Abstract eLife digest Introduction Results Discussion Materials and methods Data availability References Decision letter Author response Article and author information Metrics Abstract We hardly notice our eye blinks, yet an externally generated retinal interruption of a similar duration is perceptually salient. We examined the neural correlates of this perceptual distinction using intracranially measured ECoG signals from the human visual cortex in 14 patients. In early visual areas (V1 and V2), the disappearance of the stimulus due to either invisible blinks or salient blank video frames ('gaps') led to a similar drop in activity level, followed by a positive overshoot beyond baseline, triggered by stimulus reappearance. Ascending the visual hierarchy, the reappearance-related overshoot gradually subsided for blinks but not for gaps. By contrast, the disappearance-related drop did not follow the perceptual distinction – it was actually slightly more pronounced for blinks than for gaps. These findings suggest that blinks' limited visibility compared with gaps is correlated with suppression of blink-related visual activity transients, rather than with "filling-in" of the occluded content during blinks. https://doi.org/10.7554/eLife.17243.001 eLife digest The average person blinks once every few seconds, each time shutting off their view of the world for about a tenth of a second. Nevertheless, we rarely notice a blink. By contrast, we readily notice a single blank frame in a movie, even if the frame lasts far less than a blink. The fact that we do not usually notice our spontaneous blinks is a striking example of the discrepancy between the images we perceive versus the information that enters our eyes. This dissociation between the information that the eyes receive and what we perceive raises a number of questions. First, which brain areas represent the actual information from the eyes, and at what point do brain areas start to represent our subjective perception instead? Second, how does the brain "stabilize" our perception of vision despite the frequent interruptions that occur whenever we blink? In short, does the brain "fill in" the missing images or “edit out” the gaps? To answer these questions, Golan et al. turned to human patients who were undergoing a surgical procedure related to the treatment of epilepsy. In the course of such procedures, and strictly for diagnosis purposes, electrodes are temporarily placed directly on the surface of the brain – the cortex – making it possible to monitor the activity of individual cortical areas. Towards the back of the brain, where cortical processing of visual signals begins, neurons responded in a way that was consistent with the physical information the eye actually received rather than the perception of vision. Thus, neurons showed the same responses to easily seen blank frames in a movie as to unnoticeable blinks. However, as the signals streamed forward to down-stream brain regions involved in vision, neurons in successive areas were increasingly likely to distinguish between the perceptually visible blank frames versus the invisible blinks. Unexpectedly, Golan et al. found no evidence that the brain fills in the missing picture during blinks. Instead, it seems that the brain generates a continuous perception by actively "deleting" the brief neural signals that are turned on when our visual input has been shut off. The brain only does this for blinks but not for artificial interruptions – such as blank movie frames – which explains why we notice the latter but not the former. A future challenge will be to isolate the pathway that leads from the brain regions that generate blinks to the regions that deal with vision, and that enables us to tell blinks from blanks. https://doi.org/10.7554/eLife.17243.002 Introduction The perceived continuity of the visual input despite its frequent interruptions by spontaneous eye blinks is a ubiquitous and powerful dissociation between sensation and perception. As such, it offers an ecological test of candidate neural correlates of visual awareness. The perceptual omission of one's own eye blinks cannot be explained by the blinks' apparent briefness: blinks occlude the pupil for a considerable amount of time, typically 100–150 ms (Riggs et al., 1981). External darkenings of such duration have been shown to cause a robust percept (Riggs et al., 1981). Strikingly, since blinks normally occur at least 1000 times an hour (Cruz et al., 2011), about three or four percent of our waking hours are unknowingly spent with our eyes closed. While spontaneous blinks are invisible, voluntary blinks are not. However, their perception is also not veridical: they are experienced as shorter and less dim than physically comparable artificial darkenings (Riggs et al., 1981). Human psychophysical experiments have found evidence for a decrease in visual sensitivity during blinks even when the optical-retinal impact of blinks was neutralized (Volkmann et al., 1980). This effect could be mediated by an extra-retinal suppression of the neural response in early visual cortices during blinks, as indicated by feline V1 single unit recordings (Buisseret and Maffei, 1983) and human fMRI (Bristow et al., 2005b). Whereas the evidence for blink-related suppression of early visual activity may explain why visual sensation is reduced during blinks, it does not readily explain the perceived continuity of the visual scene across blinks. Consider a reduction of retinal input driven by an external origin, such as when someone briefly turns off the lights. Such a reduction will diminish low-level visual activity as well, yet, unlike blinks, this external reduction is clearly visible. Hence, it seems that a neural basis for the special perceptual status of blinks requires a representation that is both unperturbed by blinks and yet still sensitive to perceived external darkenings. Furthermore, if one assumes that perceived continuity relies on a read-out of an ongoing representation of the visual scene, this entails an active 'filling-in' of this visual representation during blinks but not during external darkenings (Billock, 1997). Operationally, a filling-in mechanism would be reflected in continuous neural activity across blinks but not across gaps despite the decrease in retinal input common to both. This hypothetical effect is in an opposite direction to the previously reported neuronal suppression. Critically, testing these predictions using human neuroimaging requires sufficient spatiotemporal resolution to distinguish between activity changes that occur prior, during and following the blink event across different visual regions. In particular, the sluggish BOLD-fMRI signal (used in previous related studies, e.g., Bristow et al., 2005a) can register only a temporal average of the total blink-related changes, potentially summing over antagonistic positive and negative components. Here, we overcame this limitation by examining the effect of spontaneous and voluntary eye blinks and brief external image disappearances ('gaps') on visual representations in human patients undergoing intra-cranial electrocorticographic evaluation for intractable epilepsy. This approach allowed us to test how these brief events interact with object-related human visual responses on a millisecond/millimeter-scale across multiple visual regions recorded simultaneously. Following previous findings in paradigms unrelated to blinks (Fisch et al., 2009; Moutard et al., 2015), we hypothesized that the high-frequency broadband power envelope (HFB) response of the local field potential in human high-level ventral visual cortex would reflect the perceptual distinction between extrinsic disappearance of the stimuli and their disappearance due to spontaneous eye blinks. Furthermore, we sought to directly test the intuitive yet untested conjecture that missing content due to blinks is actively filled-in by sustained neuronal activity, whereas perceived external stimulus disappearances leave 'dips' in neural responses. In brief, we found a posterior-anterior gradient in blink versus gap representations. In early visual cortex, these perceptually-distinct events elicited similar HFB responses, whereas in higher-level visual cortex, the termination of gaps elicited considerable overshoot beyond baseline levels, an effect absent in spontaneous blinks. Intriguingly, both low and high-level cortical sites failed to exhibit a differential filling-in for blinks compared to gaps, suggesting that the perceived continuity of the visual scene might not depend on a continuous neural representation in category-selective visual areas but on the lack of representation of discontinuities, that is, stimulus disappearances and reappearances. Results Fourteen patients undergoing electrocorticographic evaluation for intractable epilepsy participated in the study (see Table 1). The patients viewed consecutively presented grayscale photographs of faces and non-face images from several categories (houses, tools, abstract patterns and animals). The patients clicked the mouse button each time they detected an animal image (mean hit-rate = 86.1%, mean false-alarm rate = 3.4%). Target (animal) trials were excluded from further analysis. The images were presented at a pace of one image per second with no inter-stimulus blanks (see Figure 1 and Materials and methods). The patients were concurrently monitored for eye blinks by a video eye tracker and an electrooculogram (EOG). Once every ten images, a gray screen was displayed for three seconds, serving as a baseline and partitioning the trials into ten-trial long blocks. Critically, in some of the trials, the displayed stimuli were interrupted by either spontaneously produced eye blinks, periodic (~1 Hz) voluntary eye blinks whose production was cued at the beginning of some of the blocks, or 'gaps', produced by replacing the stimulus with a black or gray screen for variable latencies and durations, aimed at evaluating the purely retinal impact of blinks. Individual latencies and durations of the three kinds of interrupting events for each patient are presented in Figure 1—figure supplement 1. Table 1 Patients' demographic, clinical and experimental details. https://doi.org/10.7554/eLife.17243.003 Patient code*SexAgeSeizure onset zone(s)Voluntary blinks blocksBlack /gray gap controlGradual / abrupt gap controlTotal analyzed electrodesTotal visually responsive electrodesNumber of visually responsive electrodes in each ROIRetinotopicHigh-levelV1V2V3V4VOFCN-FCP20F30RH: Supramarginal Gyrus✓10360020012P25M45RH: Inferior Frontal Gyrus, Precentral S.✓11780000026P32M23RH: Superior Temporal Gyrus, Hippocampus✓181102201012P33F52LH: Hippocampus, Middle Entorhinal Cortex✓8380000041P36M24RH: Parahippocampal Gyrus, Temporal Pole**✓5880010114P39M25RH: Hippocampus, Amygdala✓✓128183221234P44M30RH: Anterior Temporal Lobe✓✓118101110111P46M45RH: Hippocampus, Parahippocampal Gyrus✓✓✓58153221214P47F34LH: Anterior Temporal Lobe✓✓✓14270001132P50M27LH: Amygdala, Hippocampus, Parahippocampal Gyrus, Anterior Fusiform Gyrus, Post Central Gyrus✓✓✓10832100000P54M21RH: Medial Temporal, Middle Occipital Gyrus, Parieto-Occipital-Sulcus, Middle Temporal Gyrus, LH: Hippocampus✓✓✓160232342020P57M29RH: Amygdala✓✓11061010002P59M50RH: Parieto-Occipital Sulcus✓✓94133000102P62F44LH: Hippocampus, Anterior Cingulate Gyrus, Amygdala, Parahippocampal Gyrus✓✓12580121000 LH/RH – left/right hemisphere, VO – ventral-occipital, FC – face-selective electrodes, N-FC non-face selective high-level electrodes. * Patients' identities were coded by order of admission to surgery. Since not every admitted patient performed the experiment, the codes are not consecutive. ** Failed to follow the instruction to voluntary blink due to language barrier. Figure 1 with 1 supplement see all Download asset Open asset Experimental design. Two blocks of the experimental task are illustrated. All stimuli are presented for 1 s each. The block on the left is preceded with an auditory instruction to execute voluntary blinks about once a second. The block on the right is interleaved with experimenter-induced black gaps (at 350, 550 or 750 ms following trial-onset). See Figure 1—figure supplement 1 for individual latency and duration distributions of gaps, voluntary blinks and spontaneous blinks. Face photographs: Owen Lucas. Available on Flickr under the Public Domain Mark 1.0 https://creativecommons.org/publicdomain/mark/1.0/). https://www.flickr.com/photos/144006675@N05/27487033282. https://www.flickr.com/photos/owen_lucas_photography/7454467582. https://www.flickr.com/photos/owen_lucas_photography/7454436922. https://www.flickr.com/photos/owen_lucas_photography/8102080226. Accessed on August 2016. https://doi.org/10.7554/eLife.17243.004 Defining visually responsive electrodes Whereas blinks may be correlated with neural responses also in non-visual regions (e.g., motor, somatosensory responses or even default mode network, see Nakano et al., 2013), in the present study we were concerned exclusively with the modulation of visual representations by blinks. Therefore, we first tested which electrodes reliably responded to the presented stimuli (e.g. faces) themselves. Since we were interested in local field potential correlates of average spiking rate, all of our analyses were conducted on the high-frequency broadband power envelope (sampled between 70 and 150 Hz). A number of previous studies have shown the HFB signal to be a good index of the aggregate firing rate (Mukamel et al., 2005; Ray et al., 2008; Rasch et al., 2008; Manning et al., 2009; Nir et al., 2007). Following standard preprocessing and HFB computation, we tested the onset response (50–350 ms, uncontaminated by either gaps or blinks) to each electrode optimal object stimuli, compared with the inter-block gray blank baseline periods (Figure 2). In agreement with previous reports (Noy et al., 2015a), responsivity of a considerable effect size was found almost entirely within the anatomically defined visual cortex, showing only minimal responses in more anterior cortical regions. Figure 2 Download asset Open asset High-frequency broadband (HFB, 70–150 Hz) visual responses to object images. HFB responses from all participants, sampled from 50 to 350 ms following the transition from one object image to another, compared with the HFB activity sampled during inter-block blanks are presented. Each circle marks the location of one electrode on a common cortical template. The response strength for each electrode's optimal (maximally responding) category is measured in standard deviations of the baseline (Glass' Δ) and is color-coded as the circles' face color saturation. Electrodes showing significant face-selectivity are presented in a red hue and the others are presented in a cyan hue. Electrodes that passed the inclusion criteria for the subsequent analyses (corrected significance < 0.05 and an effect size of at least two standard deviations) are encircled in black. The colored labels on the cortical surface were derived from a surface-based atlas of retinotopic areas (Wang et al., 2014) and from Destrieux Atlas (Destrieux et al., 2010) as implemented in FreeSurfer 5.3 (Fusiform gyrus, in red). https://doi.org/10.7554/eLife.17243.006 Figure 2—source data 1 Individual electrode data for Figure 2. Each row describes an electrode, with its SUMA standardized mesh nearest vertex (index, hemisphere and MNI coordinates), its FWE-corrected p-value for stimuli HFB greater than baseline HFB, best responding category, the effect-size of the best responding category (in baseline standard deviations), whether the electrode was selected for further analyses and face-selectivity (uncorrected) p-value. https://doi.org/10.7554/eLife.17243.007 Download elife-17243-fig2-data1-v2.csv 143 Electrodes out of a total of 1585 were found to show visual responses that were both significant (Wilcoxon rank-sum test, p<0.05, Bonferroni-corrected within-patient across electrodes) and of a considerable effect size (Glass' Δ ≥ 2, see Materials and methods). These electrodes were qualified for subsequent analyses. We chose face-preference as a model for ventral category-selectivity, motivated by its prevalent occurrence in previous recordings (Privman et al., 2007). Thus, responsive electrodes were assessed for face-selectivity by comparing the responses to face-stimuli with each non-face category. This analysis identified 19 electrodes (found in 10 of the 14 patients, see Table 1) that responded significantly more strongly to faces than to any other category (Wilcoxon rank sum test, p<0.05 for all contrasts, uncorrected), located mostly in high-level ventral visual cortex. Disentangling the effect of gaps and blinks from the stimulus-driven visual response In order to isolate the effect of interruptions (blinks and gaps) from the stimulus-driven responses, the response to the stimulus itself had to be accounted for first. We have approached this using time-domain deconvolution. This is similar to the way signals are unmixed in the analysis of fast event-related fMRI (Burock and Dale, 2000). This procedure was implemented as a multiple linear regression of the observed timecourse with a set of finite impulse response bases (see Figure 3 and Materials and methods). Its end result is estimates of the contribution of each experimental event to the observed HFB timecourse over time, after the contributions of other experimental events were accounted for. See Figure 3—figure supplement 1 for a demonstration of the advantage of this approach over standard event-related averaging. Figure 3 with 1 supplement see all Download asset Open asset A schematic illustration of the general linear model (GLM) design matrix used in the deconvolution of the neural responses. The observed time series in each electrode is modeled as a linear sum of overlapping responses triggered by the displayed stimuli and by the different interrupting events – gaps, voluntary blinks and spontaneous blinks (for simplicity, only a single set of blinks predictors appears in the illustration). Each response is composed of a sequence of non-overlapping unit pulses (4 ms-wide pulses were used for the actual 250 Hz HFB timecourse, here a less detailed, 10 Hz model is presented). Note that in this example both the stimuli and the interruptions are modeled separately for face and non-face trials. See Figure 3—figure supplement 1 for a demonstration of the advantage of this approach over standard-event related averaging. Face photographs: Owen Lucas. Available on Flickr under the Public Domain Mark 1.0 https://creativecommons.org/publicdomain/mark/1.0/). https://www.flickr.com/photos/144006675@N05/27487033282. https://www.flickr.com/photos/owen_lucas_photography/7454467582. Accessed on August 2016. https://doi.org/10.7554/eLife.17243.008 Observing gap and blink responses across the visual hierarchy Figure 4 presents examples of the gap-related, voluntary blink- and spontaneous blink-related responses in two pairs of cortical sites, each sampled within an individual patient. In both patient P46 (Figure 4a) and patient P39 (Figure 4b) there were concurrent recordings of ventral early (V1 and V2, respectively, defined by a surface-based probabilistic atlas, Wang et al., 2014) and ventral high-level (Ventral Occipital and Fusiform gyrus) visual sites. In both cases, activity in early visual sites was strongly modulated by blinks, undergoing a dip in activation levels following eye closure and then overshooting beyond expected activation levels as the eyes re-opened and the visual image reappeared. Gaps induced a qualitatively comparable effect – a negative dip in response to the onset of the display of the blank screen followed by a positive overshoot triggered by the reappearance of the stimulus. By contrast, the two higher-level ventral electrodes showed considerable reappearance-related responses following the gaps, but largely, no such responses to blinks. In P39, the high-level electrode was face-selective; by estimating blink and gap-related responses separately for face and non-face trials, we found that the reappearance-related overshoot induced by gaps (and to a lesser extent by voluntary blinks) in this electrode was face-selective as well. Figure 4 with 1 supplement see all Download asset Open asset Deconvolved high-frequency broadband responses to gaps, voluntary and spontaneous blinks, simultaneously sampled at the early visual cortex and at the ventral high-level cortex. Two pairs of electrodes from two patients are presented. All responses here are locked to the stimulus reappearance. Error bounds show the standard error of the regression coefficients. Horizontal bars mark response timepoints significantly different from zero (p<0.05, FDR-corrected within-participant, see Materials and methods). n is the number of event occurrences. Note the activation-dip followed by a reappearance-related overshoot for gaps, voluntary blinks and spontaneous blinks in early visual sites in both participants. By contrast, the two ventral high-level sites showed almost no disappearance-related dip for all three interruptions and a reappearance-related overshoot response only following gaps. (a) In participant P46, events from all trials (face and non-face alike) are depicted in black. (b) In patient P39, the high-level electrode showed greater responses to faces. By estimating the contribution of gaps, voluntary and spontaneous blinks separately during face and non-face trials (red and black traces, correspondingly), it is evident that the high-level reappearance-overshoot effect is dependent on the category of the reappearing stimulus. https://doi.org/10.7554/eLife.17243.010 Whereas most of our high-level visual electrodes in this study covered ventral regions, a similar yet distinct mode of transformation between early and late responses to blinks and gaps was evident in two cases with high-level dorsal coverage (Figure 4—figure supplement 1). Patient P57 (Figure 4—figure supplement 1a,c) had concurrent recordings in V1 and in a site in a high-level dorsal stream region, over the middle temporal gyrus, slightly less than 1 cm anterior to the border of anatomically defined MST/TO2. The dorsal high-level electrode showed a positive HFB response triggered by the image disappearance (Figure 4—figure supplement 1a, compare with 1c for image-reappearance-lock) due to gaps and no response whatsoever for blinks. By contrast, both gaps and spontaneous blinks were associated with a considerable dip in V1 activation (that particular patient was not instructed to voluntarily blink). In patient P20 (Figure 4—figure supplement 1b,d), who had adjacent electrodes in anatomically defined dorsal V3 and V3a, a sharp short-range transformation was observed. The V3 site showed a response pattern compatible with the early sites in the previous three patients, whereas the V3a site showed strong positive responses both to stimulus disappearance (Figure 4—figure supplement 1b) and to stimulus reappearance (Figure 4—figure supplement 1d) when these were caused by gaps but not when they were caused by voluntary or spontaneous blinks. Moving from these four example electrode pairs to the entire sample of fourteen patients, we parceled the visually responsive electrodes into seven regions of interest (ROIs), ensuring that each ROI contained electrodes from at least six different patients. The resulting ROIs included five retinotopic regions, V1, V2, V3, V4 and VO, defined according to the surface-based probabilistic atlas (Wang et al., 2014), face-selective electrodes (defined functionally, see above) and high-level non-face selective electrodes defined as being situated outside and away of any of the retinotopic areas specified by the surface-based atlas (Figure 5—figure supplement 1a, see Materials and methods for electrode localization and parcellation details). Figure 5a depicts the grand-averages of the deconvolved stimulus reappearance-related responses for gaps, voluntary blinks and spontaneous blinks within each ROI, separated into face and non-face trials (see Figure 5—figure supplement 1b for disappearance-locked grand-averages). Responses followed a consistent hierarchical progression: in V1 and V2, gaps, voluntary blinks and spontaneous blinks all produced qualitatively similar (but not identical) responses – which consisted of a dip in activation in response to the image disappearance, followed by an overshoot beyond baseline levels triggered by the image reappearance. Progressing along the visual hierarchy, the response to gaps and blinks diverged: the response to the image reappearance was sustained only when triggered by the termination of an external gap, and not by the termination of a blink. The negative activation dip reduced its amplitude along the hierarchy, even more so for gaps than to blinks. This is in stark contrast with the expected filling-in during blinks but not gaps. As an alternative analysis, we derived traditional event-related averages of the HFB signal of uninterrupted stimuli and subtracted them from each trial (instead of the deconvolution procedure) before computing gap and blink-related event-related-averages. This procedure yielded very similar results (Figure 5—figure supplement 1c), which rules out the possibility that the observed results were somehow an artifact of the deconvolution analysis we adopted. Figure 5b depicts the response to the object images themselves, after accounting for the effects of gaps and blinks. Note the considerable decline within several hundred milliseconds in response amplitude. In order to examine the possible effect of this decline on the HFB-dips, gap-related HFB responses were grouped by the gaps' latency relative to trial onset (Figure 5—figure supplement 2). The result of this analysis revealed similar dip magnitudes for gaps happening earlier and later in the trials in high-order regions. Figure 5 with 2 supplements see all Download asset Open asset Grand averages of deconvolved high-frequency broadband responses. (a) Responses to gaps, voluntary blinks and spontaneous blinks along the visual hierarchy. All of the traces here are locked to the image reappearance, marked as t = 0. See Figure 5—figure supplement 1b for the analogous stimulus-disappearance locked traces. After within-electrode estimation, the traces were averaged first within individuals and then across individuals, such that each grand-average is derived from independent individual traces. n is the total number of averaged electrodes for each trace. Error bounds show the standard error of the mean across individuals. Note the gradual appearance of differential responses to gaps compared with voluntary and spontaneous blink as the visual signal traveled forward. (b) Responses to object images (face and non-faces). Note that each stimulus was presented for one whole second. Face photograph: Owen Lucas. Available on Flickr under the Public Domain Mark 1.0 https://creativecommons.org/publicdomain/mark/1.0/). https://www.flickr.com/photos/144006675@N05/27487033282. Accessed on August 2016. https://doi.org/10.7554/eLife.17243.012 In order to provide a more direct visualization of the data, we generated videos depicting the reappearance-locked responses to gaps (Video 1) and spontaneous blinks (Video 2) timepoint by timepoint, across all of the 143 electrodes. Following the gaps' termination, a wave of activation that began in V1 and spread up to the anterior edge of the visual cortex, can be observed. By contrast, the blinks' termination triggered a much more localized positive activation that remained confined to V1–V3. Video 1 Download asset This video cannot be played in place because your browser does support HTML5 video. You may still download the video for offline viewing. Download as MPEG-4 Download as WebM Download as Ogg Response to gaps (stimulus-reappearance lock). Face photograph: Owen Lucas. Available on Flickr under the Public Domain Mark 1.0 https://creativecommons.org/publicdomain/mark/1.0/). https://www.flickr.com/photos/144006675@N05/27487033282. Accessed on August 2016. https://doi.org/10.7554/eLife.17243.015 Video 2 Download asset This video cannot be played in place because your browser does support HTML5 video. You may still download the video for offline viewing. Download as MPEG-4 Download as WebM Download as Ogg Response to spontaneous blinks (stimulus-reappearance lock). Face photograph: Owen Lucas. Available on Flickr under the Public Domain Mark 1.0 https://creativecommons.org/publicdomain/mark/1.0/). https://www.flickr.com/photos/144006675@N05/27487033282. Accessed on August 2016. https://doi.org/10.7554/eLife.17243.016 Quantifying single electrode negative dip and positive overshoot components In order to statistically test the reproducibility and generalizability of the observed response patterns, we quantified the two most recurring components of the gap and blink-related responses, the negative activation dip following image disappearance (interruption onset) and the positive overshoot following the image reappeara