Episodic memory, the ability to record and relive experiences, is intricately connected to visual exploration in most humans. This study explores the possibility that eye movements create physiological states relevant for memory, analogous to those associated with hippocampal theta. Previous work has demonstrated that saccadic eye movements, which occur roughly at theta frequency, elicit hippocampal event-related potentials (ERPs). Building on the Separate Phases of Encoding and Retrieval (SPEAR) model, we asked if the peaks and troughs of this saccadic ERP are differentially important for memory formation. Specifically, we applied saccade-contingent hippocampal electrical stimulation at estimated ERP peaks and troughs while individuals with epilepsy visually explored natural scenes across 59 sessions. We subsequently assessed their recognition memory for scenes and their recall of associated targets. Results indicate that memory is robust when stimulation precisely targets the peak or trough, contrasting with impairments observed with random stimulation. Moreover, memory impairment is prominent when stimulation is applied within 100 ms of saccade initiation, a time that reflects high medial temporal lobe inhibition. Our findings suggest that the hippocampus rapidly evolves through memory-relevant states following each eye movement, while also challenging the assumption that human saccadic ERP peaks and troughs mirror the encoding and retrieval phases of theta rhythms studied in rodents. The study sheds light on the dynamic interplay between eye movements, hippocampal activity, and memory formation, offering theoretical insights with potential applications for memory modulation in neurological disorders. Significance Statement Why do eye-movements enhance memory formation? Here, we causally tested if eye-movements initiate short-lived states critical for memory formation within the hippocampus, a brain region known to support memory. We built a system that could precisely apply hippocampal electrical stimulation at key moments after eye-movements to test how the timing of this stimulation influenced people’s ability to form memories. We found that stimulation was particularly disruptive to memory formation when applied within 100 ms of initiating an eye movement. By contrast, memory was robust to stimulation precisely timed to the peak and trough of hippocampal eye-movement responses. We interpret this temporal evolution of memory-relevant states within a prominent model of how theta phases relate to rodent memory. ### Competing Interest Statement David Groppe works for Persyst

Natural conversation is multisensory: when we can see the speaker's face, visual speech cues improve our comprehension. The neuronal mechanisms underlying this phenomenon remain unclear. The two main alternatives are visually mediated phase modulation of neuronal oscillations (excitability fluctuations) in auditory neurons and visual input-evoked responses in auditory neurons. Investigating this question using naturalistic audiovisual speech with intracranial recordings in humans of both sexes, we find evidence for both mechanisms. Remarkably, auditory cortical neurons track the temporal dynamics of purely visual speech using the phase of their slow oscillations and phase-related modulations in broadband high-frequency activity. Consistent with known perceptual enhancement effects, the visual phase reset amplifies the cortical representation of concomitant auditory speech. In contrast to this, and in line with earlier reports, visual input reduces the amplitude of evoked responses to concomitant auditory input. We interpret the combination of improved phase tracking and reduced response amplitude as evidence for more efficient and reliable stimulus processing in the presence of congruent auditory and visual speech inputs.SIGNIFICANCE STATEMENT Watching the speaker can facilitate our understanding of what is being said. The mechanisms responsible for this influence of visual cues on the processing of speech remain incompletely understood. We studied these mechanisms by recording the electrical activity of the human brain through electrodes implanted surgically inside the brain. We found that visual inputs can operate by directly activating auditory cortical areas, and also indirectly by modulating the strength of cortical responses to auditory input. Our results help to understand the mechanisms by which the brain merges auditory and visual speech into a unitary perception.

Event-related potentials (ERPs) are a commonly used electrophysiological signature for studying mesial temporal lobe (MTL) function during visual memory tasks. The ERPs associated with the onset of visual stimuli (image-onset) and eye movements (saccades and fixations) provide insights into the mechanisms of their generation. We hypothesized that since eye movements and image-onset provide MTL structures with salient visual information, perhaps they both engage similar neural mechanisms. To explore this question, we used intracranial electroencephalographic data from the MTLs of 11 patients with medically refractory epilepsy who participated in a visual search task. We characterized the electrophysiological responses of MTL structures to saccades, fixations, and image-onset. We demonstrated that the image-onset response is an evoked/additive response with a low-frequency power increase. In contrast, ERPs following eye movements appeared to arise from phase resetting of higher frequencies than the image-onset ERP. Intriguingly, this reset was associated with saccade onset and not termination (fixation), suggesting it is likely the MTL response to a corollary discharge, rather than a response to visual stimulation. We discuss the distinct mechanistic underpinnings of these responses which shed light on the underlying neural circuitry involved in visual memory processing.

Data-driven graph learning models a network by determining the strength of connections between its nodes. The data refers to a graph signal which associates a value with each graph node. Existing graph learning methods either use simplified models for the graph signal, or they are prohibitively expensive in terms of computational and memory requirements. This is particularly true when the number of nodes is high or there are temporal changes in the network. In order to consider richer models with a reasonable computational tractability, we introduce a graph learning method based on representation learning on graphs. Representation learning generates an embedding for each graph node, taking the information from neighbouring nodes into account. Our graph learning method further modifies the embeddings to compute the graph similarity matrix. In this work, graph learning is used to examine brain networks for brain state identification. We infer time-varying brain graphs from an extensive dataset of intracranial electroencephalographic (iEEG) signals from ten patients. We then apply the graphs as input to a classifier to distinguish seizure vs. non-seizure brain states. Using the binary classification metric of area under the receiver operating characteristic curve (AUC), this approach yields an average of 9.13 percent improvement when compared to two widely used brain network modeling methods.

Abstract Objective Epilepsy is one of the most common neurological disorders . Many individuals continue to have seizures despite medical and surgical treatments, suggesting adjunctive management strategies are required. Promising effects of daily listening to Mozart on reducing seizure frequency in individuals with epilepsy have been demonstrated over the last 20 years, but not in a rigorously controlled manner. In this study, we compared the effect on seizure frequency of daily listening to either Mozart K.448 or a spectrally similar, yet non‐rhythmic control piece. We hypothesized that there would be no difference in seizure counts when participants listened to Mozart K.448 vs when they listened to the control piece. Methods We employed a randomized crossover design, in which each participant was exposed to both three months of daily listening to the first six minutes of Sonata for two pianos in D major by Mozart (Mozart K.448; treatment period) and three months of daily listening to phase‐scrambled version (control period). There was a three‐month baseline and a three‐month follow‐up period before and after the six‐month listening period, respectively. Change in seizure counts obtained from the seizure diaries was considered as the main study outcome. Results Using three methodologies to investigate the existence of the treatment effect (paired t test, estimation statistics and plots, and Cohen's d), our results revealed a reduction in seizure counts during the treatment period, which was not observed for the control period (P‐value < .001). Significance Using a spectrally similar control piece, our study advances previous reports that were limited by a “no music” control condition. Daily listening to Mozart K.448 was associated with reducing seizure frequency in adult individuals with epilepsy. These results suggest that daily Mozart listening may be considered as an adjunctive therapeutic option to reduce seizure burden in individuals with epilepsy.

Natural conversation is multisensory: when we can see the speaker’s face, visual speech cues influence our perception of what is being said. The neuronal basis of this phenomenon remains unclear, though there is indication that phase modulation of neuronal oscillations—ongoing excitability fluctuations of neuronal populations in the brain—provides a mechanistic contribution. Investigating this question using naturalistic audiovisual speech with intracranial recordings in humans, we show that neuronal populations in auditory cortex track the temporal dynamics of unisensory visual speech using the phase of their slow oscillations and phase-related modulations in high-frequency activity. Auditory cortex thus builds a representation of the speech stream’s envelope based on visual speech alone, at least in part by resetting the phase of its ongoing oscillations. Phase reset could amplify the representation of the speech stream and organize the information contained in neuronal activity patterns. SIGNIFICANCE STATEMENT Watching the speaker can facilitate our understanding of what is being said. The mechanisms responsible for this influence of visual cues on the processing of speech remain incompletely understood. We studied those mechanisms by recording the human brain’s electrical activity through electrodes implanted surgically inside the skull. We found that some regions of cerebral cortex that process auditory speech also respond to visual speech even when it is shown as a silent movie without a soundtrack. This response can occur through a reset of the phase of ongoing oscillations, which helps augment the response of auditory cortex to audiovisual speech. Our results contribute to discover the mechanisms by which the brain merges auditory and visual speech into a unitary perception. ### Competing Interest Statement The authors have declared no competing interest.

The discovery that deep convolutional neural networks (DCNNs) achieve human performance in realistic tasks offers fresh opportunities for linking neuronal tuning properties to such tasks. Here we show that the face-space geometry, revealed through pair-wise activation similarities of face-selective neuronal groups recorded intracranially in 33 patients, significantly matches that of a DCNN having human-level face recognition capabilities. This convergent evolution of pattern similarities across biological and artificial networks highlights the significance of face-space geometry in face perception. Furthermore, the nature of the neuronal to DCNN match suggests a role of human face areas in pictorial aspects of face perception. First, the match was confined to intermediate DCNN layers. Second, presenting identity-preserving image manipulations to the DCNN abolished its correlation to neuronal responses. Finally, DCNN units matching human neuronal group tuning displayed view-point selective receptive fields. Our results demonstrate the importance of face-space geometry in the pictorial aspects of human face perception.

The unique profile of strong and weak cognitive traits characterizing each individual is of a fundamental significance, yet their neurophysiological underpinnings remain elusive. Here, we present intracranial electroencephalogram (iEEG) measurements in humans pointing to resting-state cortical "noise" as a possible neurophysiological trait that limits visual recognition capacity. We show that amplitudes of slow (<1 Hz) spontaneous fluctuations in high-frequency power measured during rest were predictive of the patients' performance in a visual recognition 1-back task (26 patients, total of 1,389 bipolar contacts pairs). Importantly, the effect was selective only to task-related cortical sites. The prediction was significant even across long (mean distance 4.6 ± 2.8 days) lags. These findings highlight the level of the individuals' internal "noise" as a trait that limits performance in externally oriented demanding tasks.

The Brain Imaging Data Structure (BIDS) is a community-driven specification for organizing neuroscience data and metadata with the aim to make datasets more transparent, reusable, and reproducible. Intracranial electroencephalography (iEEG) data offer a unique combination of high spatial and temporal resolution measurements of the living human brain. To improve internal (re)use and external sharing of these unique data, we present a specification for storing and sharing iEEG data: iEEG-BIDS.

An emerging neuromodulation approach to treat disorders such as intractable epilepsy and Alzheimer’s disease is to use electrical stimulation triggered by pathological brain states, conventionally delivered via intracranially implanted electrodes as shown in Fig. 22.8.1 (top, left). A safer, non-invasive technology for closed-loop deep-brain neuromodulation (DBN) to treat neural disorders has been a holy grail of neuroscience for decades. Recent clinical studies of non-invasive transcranial stimulation are showing increasing promise. For instance, there is evidence that stimulation in certain stages of sleep can improve memory. This work demonstrates a platform technology for enabling such studies and protocols, where brain-state-dependent stimulation enables therapy in either conventional invasive or emerging non-invasive neuromodulation modalities.

ObjectiveTo assess the variation in baseline and seizure onset zone interictal high-frequency oscillation (HFO) rates and amplitudes across different anatomic brain regions in a large cohort of patients.MethodsSeventy patients who had wide-bandwidth (5 kHz) intracranial EEG (iEEG) recordings during surgical evaluation for drug-resistant epilepsy between 2005 and 2014 who had high-resolution MRI and CT imaging were identified. Discrete HFOs were identified in 2-hour segments of high-quality interictal iEEG data with an automated detector. Electrode locations were determined by coregistering the patient's preoperative MRI with an X-ray CT scan acquired immediately after electrode implantation and correcting electrode locations for postimplant brain shift. The anatomic locations of electrodes were determined using the Desikan-Killiany brain atlas via FreeSurfer. HFO rates and mean amplitudes were measured in seizure onset zone (SOZ) and non-SOZ electrodes, as determined by the clinical iEEG seizure recordings. To promote reproducible research, imaging and iEEG data are made freely available (msel.mayo.edu).ResultsBaseline (non-SOZ) HFO rates and amplitudes vary significantly in different brain structures, and between homologous structures in left and right hemispheres. While HFO rates and amplitudes were significantly higher in SOZ than non-SOZ electrodes when analyzed regardless of contact location, SOZ and non-SOZ HFO rates and amplitudes were not separable in some lobes and structures (e.g., frontal and temporal neocortex).ConclusionsThe anatomic variation in SOZ and non-SOZ HFO rates and amplitudes suggests the need to assess interictal HFO activity relative to anatomically accurate normative standards when using HFOs for presurgical planning.

Synchronous oscillations represent a core mechanism for the temporal coordination of neural activity and the functional connectivity of distributed brain regions. A snapshot of this connectivity in time can be used to infer behavioral or pathological brain states such as the onset of a seizure. Implantable neuromodulation devices have been developed to detect such states and respond using electrical stimulation to influence neural activity, thereby preventing pathological symptoms. These devices compute biomarkers which continue to grow increasingly more complex and parameterizable. Combining the outputs of a diverse range of parameter combinations can lead to more precise brain state classification, but due to the low-power constraints of implanted devices only a subset can be extracted in real-time. The existing approach to choosing this subset experimentally is no longer practical as the selection must be chosen from millions of possible combinations. Presented here is a GPU-accelerated parameter search which automates the selection process for the commonly used phase locking value (PLV) biomarker. A GPU-based parameter extraction and elastic net approach achieves a 21.3x speedup when selecting the 200 most informative parameter combinations from a search space of over 1.2 million, resulting in a 6875x reduction in on-device processing complexity.

The advancement of implantable medical devices for the treatment of neurological disorders demands energy-efficient, low-latency processors for responsive, safe, and personalized neuromodulation. A 130-nm CMOS neural interface processor is presented to perform the brain-state classification and closed-loop control using programmable-waveform electrical stimulation. The architecture features an autoencoder neural network for both spatial filtering and dimensionality reduction. Dedicated feature extraction blocks are implemented for univariate (signal-band energy) and multivariate (phase locking value, and cross-frequency coupling) neural signal processing. The proceeding exponentially decaying memory support vector machine (EDM-SVM) accelerator employs these features for hardware-efficient brain-state classification with a high temporal resolution. An integrated digitally charge-balanced waveform generator enables flexibility in finding optimal neuromodulation paradigms for pathological symptom suppression. The system on chip (SoC) is validated using the EU human intracranial electroencephalography epilepsy data set, achieving a seizure sensitivity of 97.7% and a false detection rate of 0.185/h while consuming 169 $\mu \text{J}$ per classification.

Despite the massive accumulation of systems neuroscience findings, their functional meaning remains tentative, largely due to the absence of realistically performing models. The discovery that deep convolutional networks achieve human performance in realistic tasks offers fresh opportunities for such modeling. Here we show that the face-space topography of face-selective columns recorded intra-cranially in 32 patients significantly matches that of a DCNN having human-level face recognition capabilities. Three modeling aspects converge in pointing to a role of human face areas in pictorial rather than person identification: First, the match was confined to intermediate layers of the DCNN. Second, identity preserving image manipulations abolished the brain to DCNN correlation. Third, DCNN neurons matching face-column tuning displayed view-point selective receptive fields. Our results point to a “convergent evolution” of pattern similarities in biological and artificial face perception. They demonstrate DCNNs as a powerful modeling approach for deciphering the function of human cortical networks.

The advancement of closed-loop neuromodulation for treating neurological disorders demands: (1) analog circuits monitoring the brain activity uninterruptedly even during neurostimulation, (2) energy-efficient high-efficacy processors for responsive, adaptive, personalized neurostimulation, and (3) safe neurostimulation paradigms with rich spatio-temporal stimuli for controlling the brain's complex dynamics. This paper presents an implantable neural interface processor (NURIP) that addresses these requirements — it performs brain state classification for reliable seizure prediction and contingent seizure abortion.

Our visual environment constantly changes, yet we experience the world as a stable, unified whole. How is this stability achieved? It has been proposed that the brain preserves an implicit perceptual memory in sensory cortices [1] which stabilizes perception towards previously experienced states 2, 3. The role of higher-order areas, especially prefrontal cortex (PFC), in perceptual memory is less explored. Because PFC exhibits long neural time constants, invariance properties, and large receptive fields which may stabilize perception against time-varying inputs, it seems particularly suited to implement perceptual memory [4]. Support for this idea comes from a neuroimaging study reporting that dorsomedial PFC (dmPFC) correlates with perceptual memory [5]. But dmPFC also participates in decision making [6], so its contribution to perceptual memory could arise on a post-perceptual, decisional level [7]. To determine which role, if any, PFC plays in perceptual memory, we obtained direct intracranial recordings in six epilepsy patients while they performed sequential orientation judgements on ambiguous stimuli known to elicit perceptual memory [8]. We found that dmPFC activity in the high gamma frequency band (HGB, 70–150 Hz) correlates with perceptual memory. This effect is anatomically specific to dmPFC and functionally specific for memories of preceding percepts. Further, dmPFC appears to play a causal role, as a patient with a lesion in this area showed impaired perceptual memory. Thus, dmPFC integrates current sensory information with prior percepts, stabilizing visual experience against the perpetual variability of our surroundings.

A ubiquitous characteristic of human cortical networks is their tendency to rapidly change their response properties upon repetition. While this phenomenon has been amply documented using simple sensory-motor tasks, it is still unclear to what extent brain activations change on a short time scale when we are engaged in high level, complex tasks. Here, we examined this question using three types of high-level visual problems. We analyzed data from intracranial recordings performed in eight patients, focusing on the location and type of changes and on their relationship to overt behavior. Our results show significant repetition effects, manifested as signal decrease with repetition, in three different groups of electrodes: Visual sites, which increased their activity during stimuli presentation; Processing Positive sites, which demonstrated increased activity throughout the experimental trial; and Processing Negative sites, which demonstrated suppression of activity during the trial as compared to baseline. Interestingly, despite these significant repetition effects, response time remained unchanged with repetition. These findings bear directly upon our ability to interpret results aggregated across multiple repetitions of the same complex task.

Many environmental stimuli contain temporal regularities, a feature that can help predict forthcoming input. Phase locking (entrainment) of ongoing low-frequency neuronal oscillations to rhythmic stimuli is proposed as a potential mechanism for enhancing neuronal responses and perceptual sensitivity, by aligning high-excitability phases to events within a stimulus stream. Previous experiments show that rhythmic structure has a behavioral benefit even when the rhythm itself is below perceptual detection thresholds (ten Oever et al., 2014). It is not known whether this “inaudible” rhythmic sound stream also induces entrainment. Here we tested this hypothesis using magnetoencephalography and electrocorticography in humans to record changes in neuronal activity as subthreshold rhythmic stimuli gradually became audible. We found that significant phase locking to the rhythmic sounds preceded participants' detection of them. Moreover, no significant auditory-evoked responses accompanied this prethreshold entrainment. These auditory-evoked responses, distinguished by robust, broad-band increases in intertrial coherence, only appeared after sounds were reported as audible. Taken together with the reduced perceptual thresholds observed for rhythmic sequences, these findings support the proposition that entrainment of low-frequency oscillations serves a mechanistic role in enhancing perceptual sensitivity for temporally predictive sounds. This framework has broad implications for understanding the neural mechanisms involved in generating temporal predictions and their relevance for perception, attention, and awareness. SIGNIFICANCE STATEMENT The environment is full of rhythmically structured signals that the nervous system can exploit for information processing. Thus, it is important to understand how the brain processes such temporally structured, regular features of external stimuli. Here we report the alignment of slowly fluctuating oscillatory brain activity to external rhythmic structure before its behavioral detection. These results indicate that phase alignment is a general mechanism of the brain to process rhythmic structure and can occur without the perceptual detection of this temporal structure.