The auditory brainstem response (ABR) is a measure of subcortical activity in response to auditory stimuli. The wave V peak of the ABR depends on the stimulus intensity level, and has been widely used for clinical hearing assessment. Conventional methods estimate the ABR average electroencephalography (EEG) responses to short unnatural stimuli such as clicks. Recent work has moved toward more ecologically relevant continuous speech stimuli using linear deconvolution models called temporal response functions (TRFs). Investigating whether the TRF waveform changes with stimulus intensity is a crucial step toward the use of natural speech stimuli for hearing assessments involving subcortical responses. Here, we develop methods to estimate level-dependent subcortical TRFs using EEG data collected from 21 participants listening to continuous speech presented at 4 different intensity levels. We find that level-dependent changes can be detected in the wave V peak of the subcortical TRF for almost all participants, and are consistent with level-dependent changes in click-ABR wave V. We also investigate the most suitable peripheral auditory model to generate predictors for level-dependent subcortical TRFs and find that simple gammatone filterbanks perform the best. Additionally, around 6 min of data may be sufficient for detecting level-dependent effects and wave V peaks above the noise floor for speech segments with higher intensity. Finally, we show a proof-of-concept that level-dependent subcortical TRFs can be detected even for the inherent intensity fluctuations in natural continuous speech.

The auditory brainstem response (ABR) is a valuable clinical tool for objective hearing assessment, which is conventionally detected by averaging neural responses to thousands of short stimuli. Progressing beyond these unnatural stimuli, brainstem responses to continuous speech presented via earphones have been recently detected using linear temporal response functions (TRFs). Here, we extend earlier studies by measuring subcortical responses to continuous speech presented in the sound-field, and assess the amount of data needed to estimate brainstem TRFs. Electroencephalography (EEG) was recorded from 24 normal hearing participants while they listened to clicks and stories presented via earphones and loudspeakers. Subcortical TRFs were computed after accounting for non-linear processing in the auditory periphery by either stimulus rectification or an auditory nerve model. Our results demonstrated that subcortical responses to continuous speech could be reliably measured in the sound-field. TRFs estimated using auditory nerve models outperformed simple rectification, and 16 minutes of data was sufficient for the TRFs of all participants to show clear wave V peaks for both earphones and sound-field stimuli. Subcortical TRFs to continuous speech were highly consistent in both earphone and sound-field conditions, and with click ABRs. However, sound-field TRFs required slightly more data (16 minutes) to achieve clear wave V peaks compared to earphone TRFs (12 minutes), possibly due to effects of room acoustics. By investigating subcortical responses to sound-field speech stimuli, this study lays the groundwork for bringing objective hearing assessment closer to real-life conditions, which may lead to improved hearing evaluations and smart hearing technologies.

Perception of sounds and speech involves structures in the auditory brainstem that rapidly process ongoing auditory stimuli. The role of these structures in speech processing can be investigated by measuring their electrical activity using scalp-mounted electrodes. However, typical analysis methods involve averaging neural responses to many short repetitive stimuli that bear little relevance to daily listening environments. Recently, subcortical responses to more ecologically relevant continuous speech were detected using linear encoding models. These methods estimate the temporal response function (TRF), which is a regression model that minimises the error between the measured neural signal and a predictor derived from the stimulus. Using predictors that model the highly non-linear peripheral auditory system may improve linear TRF estimation accuracy and peak detection. Here, we compare predictors from both simple and complex peripheral auditory models for estimating brainstem TRFs on electroencephalography (EEG) data from 24 participants listening to continuous speech. We also investigate the data length required for estimating subcortical TRFs, and find that around 12 minutes of data is sufficient for clear wave V peaks (>3 dB SNR) to be seen in nearly all participants. Interestingly, predictors derived from simple filterbank-based models of the peripheral auditory system yield TRF wave V peak SNRs that are not significantly different from those estimated using a complex model of the auditory nerve, provided that the nonlinear effects of adaptation in the auditory system are appropriately modelled. Crucially, computing predictors from these simpler models is more than 50 times faster compared to the complex model. This work paves the way for efficient modelling and detection of subcortical processing of continuous speech, which may lead to improved diagnosis metrics for hearing impairment and assistive hearing technology.

Hearing-impaired listeners have a reduced ability to selectively attend to sounds of interest amid distracting sounds in everyday environments. This ability is not fully regained with modern hearing technology. A better understanding of the brain mechanisms underlying selective attention during speech processing may lead to brain-controlled hearing aids with improved detection and amplification of the attended speech. Prior work has shown that brain responses to speech, measured with magnetoencephalography (MEG) or electroencephalography (EEG), are modulated by selective attention. These responses can be predicted from the speech signal through linear filters called Temporal Response Functions (TRFs). Unfortunately, these sensor-level predictions are often noisy and do not provide much insight into specific brain source locations. Therefore, a novel method called Neuro-Current Response Functions (NCRFs) was recently introduced to directly estimate linear filters at the brain source level from MEG responses to speech from one talker. However, MEG is not well-suited for wearable and realtime hearing technologies. This work aims to adapt the NCRF method for EEG under more realistic listening environments. EEG data was recorded from a hearing-impaired listener while attending to one of two competing talkers embedded in 16-talker babble noise. Preliminary results indicate that source-localized linear filters can be directly estimated from EEG data in such competing-talker scenarios. Future work will focus on evaluating the current method on a larger dataset and on developing novel methods, which may aid in the improvement of next-generation brain-controlled hearing technology.

Even though human experience unfolds continuously in time, it is not strictly linear; instead, it entails cascading processes building hierarchical cognitive structures. For instance, during speech perception, humans transform a continuously varying acoustic signal into phonemes, words, and meaning, and these levels all have distinct but interdependent temporal structures. Time-lagged regression using temporal response functions (TRFs) has recently emerged as a promising tool for disentangling electrophysiological brain responses related to such complex models of perception. Here, we introduce the Eelbrain Python toolkit, which makes this kind of analysis easy and accessible. We demonstrate its use, using continuous speech as a sample paradigm, with a freely available EEG dataset of audiobook listening. A companion GitHub repository provides the complete source code for the analysis, from raw data to group-level statistics. More generally, we advocate a hypothesis-driven approach in which the experimenter specifies a hierarchy of time-continuous representations that are hypothesized to have contributed to brain responses, and uses those as predictor variables for the electrophysiological signal. This is analogous to a multiple regression problem, but with the addition of a time dimension. TRF analysis decomposes the brain signal into distinct responses associated with the different predictor variables by estimating a multivariate TRF (mTRF), quantifying the influence of each predictor on brain responses as a function of time(-lags). This allows asking two questions about the predictor variables: (1) Is there a significant neural representation corresponding to this predictor variable? And if so, (2) what are the temporal characteristics of the neural response associated with it? Thus, different predictor variables can be systematically combined and evaluated to jointly model neural processing at multiple hierarchical levels. We discuss applications of this approach, including the potential for linking algorithmic/representational theories at different cognitive levels to brain responses through computational models with appropriate linking hypotheses.

Auditory cortical responses to speech obtained by magnetoencephalography (MEG) show robust speech tracking to the speaker's fundamental frequency in the high-gamma band (70-200 Hz), but little is currently known about whether such responses depend on the focus of selective attention. In this study 22 human subjects listened to concurrent, fixed-rate, speech from male and female speakers, and were asked to selectively attend to one speaker at a time, while their neural responses were recorded with MEG. The male speaker's pitch range coincided with the lower range of the high-gamma band, whereas the female speaker's higher pitch range had much less overlap, and only at the upper end of the high-gamma band. Neural responses were analyzed using the temporal response function (TRF) framework. As expected, the responses demonstrate robust speech tracking of the fundamental frequency in the high-gamma band, but only to the male's speech, with a peak latency of approximately 40 ms. Critically, the response magnitude depends on selective attention: the response to the male speech is significantly greater when male speech is attended than when it is not attended, under acoustically identical conditions. This is a clear demonstration that even very early cortical auditory responses are influenced by top-down, cognitive, neural processing mechanisms.

Cortical ischaemic strokes result in cognitive deficits depending on the area of the affected brain. However, we have demonstrated that difficulties with attention and processing speed can occur even with small subcortical infarcts. Symptoms appear independent of lesion location, suggesting they arise from generalized disruption of cognitive networks. Longitudinal studies evaluating directional measures of functional connectivity in this population are lacking. We evaluated six patients with minor stroke exhibiting cognitive impairment 6-8 weeks post-infarct and four age-similar controls. Resting-state magnetoencephalography data were collected. Clinical and imaging evaluations of both groups were repeated 6- and 12 months later. Network Localized Granger Causality was used to determine differences in directional connectivity between groups and across visits, which were correlated with clinical performance. Directional connectivity patterns remained stable across visits for controls. After the stroke, inter-hemispheric connectivity between the frontoparietal cortex and the non-frontoparietal cortex significantly increased between visits 1 and 2, corresponding to uniform improvement in reaction times and cognitive scores. Initially, the majority of functional links originated from non-frontal areas contralateral to the lesion, connecting to ipsilesional brain regions. By visit 2, inter-hemispheric connections, directed from the ipsilesional to the contralesional cortex significantly increased. At visit 3, patients demonstrating continued favourable cognitive recovery showed less reliance on these inter-hemispheric connections. These changes were not observed in those without continued improvement. Our findings provide supporting evidence that the neural basis of early post-stroke cognitive dysfunction occurs at the network level, and continued recovery correlates with the evolution of inter-hemispheric connectivity.

Neural speech tracking has advanced our understanding of how our brains rapidly map an acoustic speech signal onto linguistic representations and ultimately meaning. It remains unclear, however, how speech intelligibility is related to the corresponding neural responses. Many studies addressing this question vary the level of intelligibility by manipulating the acoustic waveform, but this makes it difficult to cleanly disentangle the effects of intelligibility from underlying acoustical confounds. Here, using magnetoen-cephalography recordings, we study neural measures of speech intelligibility by manipu-lating intelligibility while keeping the acoustics strictly unchanged. Acoustically identical degraded speech stimuli (three -band noise- vocoded, -20 s duration) are presented twice, but the second presentation is preceded by the original (nondegraded) version of the speech. This intermediate priming, which generates a "pop -out" percept, substantially improves the intelligibility of the second degraded speech passage. We investigate how intelligibility and acoustical structure affect acoustic and linguistic neural representa-tions using multivariate temporal response functions (mTRFs). As expected, behavioral results confirm that perceived speech clarity is improved by priming. mTRFs analysis reveals that auditory (speech envelope and envelope onset) neural representations are not affected by priming but only by the acoustics of the stimuli (bottom-up driven). Critically, our findings suggest that segmentation of sounds into words emerges with better speech intelligibility, and most strongly at the later (-400 ms latency) word pro-cessing stage, in prefrontal cortex, in line with engagement of top-down mechanisms associated with priming. Taken together, our results show that word representations may provide some objective measures of speech comprehension.

OBJECTIVE:The Temporal Response Function (TRF) is a linear model of neural activity time-locked to continuous stimuli, including continuous speech. TRFs based on speech envelopes typically have distinct components that have provided remarkable insights into the cortical processing of speech. However, current methods may lead to less than reliable estimates of single-subject TRF components. Here, we compare two established methods, in TRF component estimation, and also propose novel algorithms that utilize prior knowledge of these components, bypassing the full TRF estimation. METHODS:We compared two established algorithms, ridge and boosting, and two novel algorithms based on Subspace Pursuit (SP) and Expectation Maximization (EM), which directly estimate TRF components given plausible assumptions regarding component characteristics. Single-channel, multi-channel, and source-localized TRFs were fit on simulations and real magnetoencephalographic data. Performance metrics included model fit and component estimation accuracy. RESULTS:Boosting and ridge have comparable performance in component estimation. The novel algorithms outperformed the others in simulations, but not on real data, possibly due to the plausible assumptions not actually being met. Ridge had slightly better model fits on real data compared to boosting, but also more spurious TRF activity. CONCLUSION:Results indicate that both smooth (ridge) and sparse (boosting) algorithms perform comparably at TRF component estimation. The SP and EM algorithms may be accurate, but rely on assumptions of component characteristics. SIGNIFICANCE:This systematic comparison establishes the suitability of widely used and novel algorithms for estimating robust TRF components, which is essential for improved subject-specific investigations into the cortical processing of speech.

Stroke patients with hemiparesis display decreased beta band (13-25 Hz) rolandic activity, correlating to impaired motor function. However, clinically, patients without significant weakness, with small lesions far from sensorimotor cortex, exhibit bilateral decreased motor dexterity and slowed reaction times. We investigate whether these minor stroke patients also display abnormal beta band activity. Magnetoencephalographic (MEG) data were collected from nine minor stroke patients (NIHSS < 4) without significant hemiparesis, at ~1 and ~6 months postinfarct, and eight age-similar controls. Rolandic relative beta power during matching tasks and resting state, and Beta Event Related (De)Synchronization (ERD/ERS) during button press responses were analyzed. Regardless of lesion location, patients had significantly reduced relative beta power and ERS compared to controls. Abnormalities persisted over visits, and were present in both ipsi- and contra-lesional hemispheres, consistent with bilateral impairments in motor dexterity and speed. Minor stroke patients without severe weakness display reduced rolandic beta band activity in both hemispheres, which may be linked to bilaterally impaired dexterity and processing speed, implicating global connectivity dysfunction affecting sensorimotor cortex independent of lesion location. Findings not only illustrate global network disruption after minor stroke, but suggest rolandic beta band activity may be a potential biomarker and treatment target, even for minor stroke patients with small lesions far from sensorimotor areas.

Primary auditory cortex is a critical stage in the human auditory pathway, a gateway between subcortical and higher-level cortical areas. Receiving the output of all subcortical processing, it sends its output on to higher-level cortex. Non-invasive physiological recordings of primary auditory cortex using electroencephalography (EEG) and magnetoencephalography (MEG), however, may not have sufficient specificity to separate responses generated in primary auditory cortex from those generated in underlying subcortical areas or neighboring cortical areas. This limitation is important for investigations of effects of top-down processing (e.g., selective-attention-based) on primary auditory cortex: higher-level areas are known to be strongly influenced by top-down processes, but subcortical areas are often assumed to perform strictly bottom-up processing. Fortunately, recent advances have made it easier to isolate the neural activity of primary auditory cortex from other areas. In this perspective, we focus on time-locked responses to stimulus features in the high gamma band (70–150 Hz) and with early cortical latency (∼40 ms), intermediate between subcortical and higher-level areas. We review recent findings from physiological studies employing either repeated simple sounds or continuous speech, obtaining either a frequency following response (FFR) or temporal response function (TRF). The potential roles of top-down processing are underscored, and comparisons with invasive intracranial EEG (iEEG) and animal model recordings are made. We argue that MEG studies employing continuous speech stimuli may offer particular benefits, in that only a few minutes of speech generates robust high gamma responses from bilateral primary auditory cortex, and without measurable interference from subcortical or higher-level areas.

Cortical processing of arithmetic and of language rely on both shared and task-specific neural mechanisms, which should also be dissociable from the particular sensory modality used to probe them. Here, spoken arithmetical and non-mathematical statements were employed to investigate neural processing of arithmetic, compared with general language processing, in an attention-modulated cocktail party paradigm. Magnetoencephalography (MEG) data were recorded from 22 human subjects listening to audio mixtures of spoken sentences and arithmetic equations while selectively attending to one of the two speech streams. Short sentences and simple equations were presented diotically at fixed and distinct word/symbol and sentence/equation rates. Critically, this allowed neural responses to acoustics, words, and symbols to be dissociated from responses to sentences and equations. Indeed, the simultaneous neural processing of the acoustics of words and symbols was observed in auditory cortex for both streams. Neural responses to sentences and equations, however, were predominantly to the attended stream, originating primarily from left temporal, and parietal areas, respectively. Additionally, these neural responses were correlated with behavioral performance in a deviant detection task. Source-localized temporal response functions (TRFs) revealed distinct cortical dynamics of responses to sentences in left temporal areas and equations in bilateral temporal, parietal, and motor areas. Finally, the target of attention could be decoded from MEG responses, especially in left superior parietal areas. In short, the neural responses to arithmetic and language are especially well segregated during the cocktail party paradigm, and the correlation with behavior suggests that they may be linked to successful comprehension or calculation. SIGNIFICANCE STATEMENT Neural processing of arithmetic relies on dedicated, modality independent cortical networks that are distinct from those underlying language processing. Using a simultaneous cocktail party listening paradigm, we found that these separate networks segregate naturally when listeners selectively attend to one type over the other. Neural responses in the left temporal lobe were observed for both spoken sentences and equations, but the latter additionally showed bilateral parietal activity consistent with arithmetic processing. Critically, these responses were modulated by selective attention and correlated with task behavior, consistent with reflecting high-level processing for speech comprehension or correct calculations. The response dynamics show task-related differences that were used to reliably decode the attentional target of sentences or equations.

Stroke patients with small central nervous system infarcts often demonstrate an acute dysexecutive syndrome characterized by difficulty with attention, concentration, and processing speed, independent of lesion size or location. We use magnetoencephalography (MEG) to show that disruption of network dynamics may be responsible. Nine patients with recent minor strokes and eight age-similar controls underwent cognitive screening using the Montreal cognitive assessment (MoCA) and MEG to evaluate differences in cerebral activation patterns. During MEG, subjects participated in a visual picture-word matching task. Task complexity was increased as testing progressed. Cluster-based permutation tests determined differences in activation patterns within the visual cortex, fusiform gyrus, and lateral temporal lobe. At visit 1, MoCA scores were significantly lower for patients than controls (median [interquartile range] = 26.0 [4] versus 29.5 [3], P = 0.005), and patient reaction times were increased. The amplitude of activation was significantly lower after infarct and demonstrated a pattern of temporal dispersion independent of stroke location. Differences were prominent in the fusiform gyrus and lateral temporal lobe. The pattern suggests that distributed network dysfunction may be responsible. Additionally, controls were able to modulate their cerebral activity based on task difficulty. In contrast, stroke patients exhibited the same low-amplitude response to all stimuli. Group differences remained, to a lesser degree, 6 mo later; while MoCA scores and reaction times improved for patients. This study suggests that function is a globally distributed property beyond area-specific functionality and illustrates the need for longer-term follow-up studies to determine whether abnormal activation patterns ultimately resolve or another mechanism underlies continued recovery.

Neural processing along the ascending auditory pathway is often associated with a progressive reduction in characteristic processing rates. For instance, the well-known frequency-following response (FFR) of the auditory midbrain, as measured with electroencephalography (EEG), is dominated by frequencies from ∼100 Hz to several hundred Hz, phase-locking to the acoustic stimulus at those frequencies. In contrast, cortical responses, whether measured by EEG or magnetoencephalography (MEG), are typically characterized by frequencies of a few Hz to a few tens of Hz, time-locking to acoustic envelope features. In this study we investigated a crossover case, cortically generated responses time-locked to continuous speech features at FFR-like rates. Using MEG, we analyzed responses in the high gamma range of 70–200 Hz to continuous speech using neural source-localized reverse correlation and the corresponding temporal response functions (TRFs). Continuous speech stimuli were presented to 40 subjects (17 younger, 23 older adults) with clinically normal hearing and their MEG responses were analyzed in the 70–200 Hz band. Consistent with the relative insensitivity of MEG to many subcortical structures, the spatiotemporal profile of these response components indicated a cortical origin with ∼40 ms peak latency and a right hemisphere bias. TRF analysis was performed using two separate aspects of the speech stimuli: a) the 70–200 Hz carrier of the speech, and b) the 70–200 Hz temporal modulations in the spectral envelope of the speech stimulus. The response was dominantly driven by the envelope modulation, with a much weaker contribution from the carrier. Age-related differences were also analyzed to investigate a reversal previously seen along the ascending auditory pathway, whereby older listeners show weaker midbrain FFR responses than younger listeners, but, paradoxically, have stronger cortical low frequency responses. In contrast to both these earlier results, this study did not find clear age-related differences in high gamma cortical responses to continuous speech. Cortical responses at FFR-like frequencies shared some properties with midbrain responses at the same frequencies and with cortical responses at much lower frequencies.

We consider the problem of determining Granger causal influences among sources that are indirectly observed through low-dimensional and noisy linear projections. Commonly used methods proceed in a two-stage fashion, by first solving an inverse problem to localize the sources, and then inferring the Granger causal influences from the estimated sources. The inferred Granger causal links thus inherit the various biases that are used in source localization techniques, in the form of spatiotemporal priors designed in favor of spatial localization. In addition, this approach does not account for the structural properties of the underlying functional networks such as sparsity of the links. We address these issues by modeling the source dynamics as sparse vector autoregressive processes and estimate the model parameters directly from the observations, with no recourse to intermediate source localization. We evaluate the performance of the proposed methodology using both simulated and experimentally-recorded MEG data.

1Department of Electrical and Computer Engineering, University of Maryland, College Park, Maryland, USA 2Institute for Systems Research, University of Maryland, College Park, Maryland, USA 3Audiology and Speech Pathology Center, Walter Reed National Military Medical Center, Bethesda, Maryland, USA 4Department of Hearing and Speech Sciences, University of Maryland, College Park, Maryland, USA 5Department of Biology, University of Maryland, College Park, Maryland, USA

The P300 wave is an Event Related Potential that is elicited in the brain when a subject is presented with a familiar stimulus. The Guilty Knowledge Test is used to determine if certain information is stored in the brain by detecting the P300 wave. In this paper, an improved Guilty Knowledge Test has been conducted on 14 subjects. Although a variety of feature extraction and classification algorithms for detecting the P300 have been used thus far, this paper introduces the use of deep learning techniques for the Guilty Knowledge Test. Two deep learning techniques; Deep Belief Networks and Stacked Autoencoders, have been compared. The input to these classifiers was the filtered EEG signal without any feature extraction. A mean accuracy of 86.9% was achieved for the Deep Belief Network and 86.01% for the Stacked Autoencoder on our dataset, outperforming the conventional Support Vector Machine.

The P300 Speller is a Brain Computer Interface that enables communication using the EEG signal. The P300 wave is an Event Related Potential that occurs as a response to a familiar stimulus. This system can be used to aid persons who are unable to communicate via conventional methods. In this paper, the P300 Speller has been modified to allow communication in three languages: English, Sinhala and Tamil. Classification of the P300 waveform was done using an ensemble of Support Vector Machines. A mean letter classification accuracy of 83.76% and a mean P300 classification accuracy of 90.68% was achieved on our dataset.