Statistical learning shapes neural sequence representations

Although sensory input arrives continuously, we experience the world in discrete units, consisting of words, objects, scenes and events, which in turn form the basis of language, thought and memory. How does sensory input get parsed into meaningful units? A process known as statistical learning (SL) may underlie this ability. SL is ubiquitous, for example allowing babies to discover word boundaries in continuous speech by tracking transitional probabilities between syllables. Here we examine which cortical circuits extract such regularities, how these regularities are represented, and how this learning compares across sensory modalities. We exposed subjects to auditory and visual sequences containing temporal regularities while collecting direct, intracranial recordings (23 patients, 3689 electrodes). We used neural frequency tagging to first map the cortical circuits for SL and then representational similarity analysis to determine which aspect(s) of the regularities are learned. SL manifested into two distinct ways across electrodes, differing in terms of both anatomical location and hierarchical organization: one cluster of electrodes located in earlier processing stages (e.g., superior temporal gyrus, STG) represented both the constituent elements (e.g., syllables) and learned higher-order units (e.g., words); the other cluster was localized to later processing stages (e.g., inferior frontal gyrus, IFG) and represented only the higher-order units. Within these regions, SL shaped the similarity of neural representations at multiple levels, with a division of labor between earlier vs. later brain areas in terms of encoding of simple generic aspects of the sequences (i.e., transitional probability) vs. complex and specific information (i.e., ordinal position, identity). The anatomical and representational segregation of these circuits was observed for SL in both the auditory and visual modality, yet the anatomical areas (with the exception of IFG and anterior temporal pole), showed specificity to modality. These findings indicate the existence of multiple computational systems for sequence processing supporting learning across hierarchically segregated cortical circuits.