Our understanding of visual cortical processing has relied primarily on studying the selectivity of individual neurons in different areas. A complementary approach is to study how the representational geometry of neuronal populations differs across areas, which can reveal encoding strategies difficult to infer from single neuron responses. We measured neuronal population responses in primary visual cortex (V1) and area V2 of macaque monkeys to naturalistic textures. Responses were lower dimensional in V2 than V1, and there was a better alignment among V2 population responses to different textures. The representational geometry in V2 afforded better discriminability between out-of-sample textures. We performed complementary analyses of standard convolutional network models, which did not replicate the representational geometry of cortex. We conclude that from V1 to V2 the visual representation changes to emphasize a low-dimensional, systematic encoding of different textures and of different instantiations of each texture.

Tracking signal flow across areas is essential for understanding brain function. Recent studies using cross-covariances show that activity directionality can shift rapidly with behavioral or task demands; yet, the circuit mechanisms underlying these changes remain unclear. Here, we use recurrent network models to investigate how directional interactions emerge and are flexibly reconfigured in multi-area cortical circuits. We show that, for fixed connectivity, directionality is shaped by how common inputs align with recurrent connectivity and the associated internal timescales of activity. In multi-area circuits with locally balanced excitation and inhibition, this reveals a predominant role for inputs to excitatory over inhibitory populations in controlling directionality. These inputs govern the directionality of the latent signals that account for most of the shared activity across areas, predominantly reflecting widespread and coherent activity fluctuations. Our models capture key features of cross-covariances from primate areas V1 and V2 and suggest parsimonious mechanisms for the shift in directionality reported in these areas. This work establishes a mechanistic framework for understanding dynamic changes in signal flow between brain areas.

Summary Response variability is pervasive in sensory cortex: neuronal activity varies across repeated presentations of the same sensory input. The properties of this response variability have been characterized extensively. One of its most well-documented features is that variability is often shared or correlated between neurons: when one neuron fires more vigorously than its typical response, other neurons are likely to do so as well. Given the prevalence of correlated variability, much work has been devoted to understanding its functional consequences. Theoretical and computational work has proven indispensable for generating hypotheses about how cortical variability affects neural coding and, ultimately, perception and behavior. Two different views of correlated variability have arisen from this work. First, variability has been considered as noise which can limit the accuracy of the neural encoding of task-relevant variables. Second, subsequent studies proposed that variability is instead a constitutive element of the neural code, representing the uncertainty with which a variable is encoded. These ideas have been formalized using information-theoretic measures and computational models, which has allowed for a systematic examination of the benefits and drawbacks of different forms of correlations. Furthermore, several experiments have been conducted to test these theoretical proposals, offering a solid foundation for future studies of the neural code. Building on this foundation, ongoing research is expanding our view of the role of structured variability in cortical function.

Gaussian processes are now commonly used in dimensionality reduction approaches tailored to neuroscience, especially to describe changes in high-dimensional neural activity over time. As recording capabilities expand to include neuronal populations across multiple brain areas, cortical layers, and cell types, interest in extending gaussian process factor models to characterize multipopulation interactions has grown. However, the cubic runtime scaling of current methods with the length of experimental trials and the number of recorded populations (groups) precludes their application to large-scale multipopulation recordings. Here, we improve this scaling from cubic to linear in both trial length and group number. We present two approximate approaches to fitting multigroup gaussian process factor models based on inducing variables and the frequency domain. Empirically, both methods achieved orders of magnitude speed-up with minimal impact on statistical performance, in simulation and on neural recordings of hundreds of neurons across three brain areas. The frequency domain approach, in particular, consistently provided the greatest runtime benefits with the fewest trade-offs in statistical performance. We further characterize the estimation biases introduced by the frequency domain approach and demonstrate effective strategies to mitigate them. This work enables a powerful class of analysis techniques to keep pace with the growing scale of multipopulation recordings, opening new avenues for exploring brain function.

Determining how the brain encodes sensory information requires understanding the structure of cortical activity, including how its variability is shared among neurons. The role of this covariability in cortical representations of natural visual inputs is unclear. Here, we adopt the neural sampling hypothesis and extend a well-established generative model of image statistics, to explain pairwise activity as representing joint probabilistic inferences about latent features of images. According to the theory, variability reflects uncertainty about those latent features. In natural images, some sources of uncertainty are shared between features and lead to covariability between neurons, whereas other independent sources contribute to private variability. Our analysis shows that spatial context in images reduces shared uncertainty for overlapping features, whereas it reduces independent uncertainty for non-overlapping features. As a result, the model predicts that increasing the size of an image reduces correlations for pairs with overlapping receptive fields and increases correlations for pairs with offset receptive fields. This prediction was confirmed by recordings from male macaque primary visual cortex (V1). Our study establishes a precise connection between V1 correlations and natural scene statistics, suggesting patterns of covariability are a feature of probabilistic representations of scenes.

Adaptation affects neuronal responsivity and selectivity throughout the visual hierarchy. However, because most prior studies have tailored stimuli to a single brain area of interest, we have a poor understanding of how exposure to a particular image alters responsivity and tuning at different stages of visual processing. Here we assess how adaptation with naturalistic textures alters neuronal responsivity and selectivity in primary visual cortex (V1) and area V2 of macaque monkeys. Neurons in both areas respond to textures, but V2 neurons are sensitive to higher-order image statistics which do not strongly modulate V1 responsivity. We tested the specificity of adaptation in each area with textures and spectrally matched "noise" stimuli. Adaptation reduced responsivity in both V1 and V2, but only in V2 was the reduction dependent on the presence of higher-order texture statistics. Despite this specificity, the texture information provided by single neurons and populations was reduced after adaptation, in both V1 and V2. Our results suggest that adaptation effects for a given feature are induced at the stage of processing that tuning for that feature first arises and that stimulus-specific adaptation effects need not result in improved sensory encoding.

The current study investigated how the brain sets up expectations from stimulus regularities by evaluating the neural responses to expectations driven implicitly (by the stimuli themselves) and explicitly (by task demands). How the brain uses prior information to create expectations and what role attention plays in forming or holding predictions to efficiently respond to incoming sensory information is still debated. We presented temporal patterns of visual input while recording EEG under two different task conditions. When the patterns were task-relevant and pattern recognition was required to perform the button press task, three different event-related brain potentials (ERPs) were elicited, each reflecting a different aspect of pattern expectation. In contrast, when the patterns were task-irrelevant, none of the neural indicators of pattern recognition or pattern violation detection were observed to the same temporally structured sequences. Thus, results revealed a clear distinction between expectation and attention that was prompted by task requirements. These results provide complementary pieces of evidence that implicit exposure to a stimulus pattern may not be sufficient to drive neural effects of expectations that lead to predictive error responses. Task-driven attentional control can dissociate from stimulus-driven expectations, to effectively minimize distracting information and maximize attentional regulation.

Correlated variability in the visual cortex is modulated by stimulus properties. The stimulus dependence of correlated variability impacts stimulus coding and is indicative of circuit structure. An affine model combining a multiplicative factor and an additive offset has been proposed to explain how correlated variability in primary visual cortex (V1) depends on stimulus orientations. However, whether the affine model could be extended to explain modulations by other stimulus variables or variability shared between two brain areas is unknown. Motivated by a simple neural circuit mechanism, we modified the affine model to better explain the contrast-dependence of neural variability shared within either primary or secondary visual cortex (V1 or V2) as well as the orientation-dependence of neural variability shared between V1 and V2. Our results bridge neural circuit mechanisms and statistical models, and provide a parsimonious explanation for the stimulus-dependence of correlated variability within and between visual areas.

Salience is critical to vision. It allows stimuli that are different from their surroundings to 'pop out', drawing our attention. Perceptual salience is postulated to be encoded via a saliency map, based on differences in neuronal responsivity to simple image features at different spatial locations. Simple image features such as luminance, orientation and color are known to affect saliency and many of these features are encoded in primary visual cortex (V1), which several influential theories propose instantiate a saliency map. However, the degree to which more complex image features can determine salience, and whether there are neural correlates of salience which are computed outside of V1, remains unclear. Here we use displays of naturalistic textures to test for neural correlates of salience-termed pop-out responses-in V1 and area V2 of anesthetized macaque monkeys. Sensitivity to higher-order texture statistics arises in V2, so pop-out responses for these displays, if they exist, would be expected to be computed after V1. We presented displays in which a target texture, presented within the neuronal receptive field, was surrounded by distractors. Distractors could differ from the target texture in either higher-order texture statistics only, or in both lower- and higher-order statistics. We found little evidence for pop-out signals in either V1 or V2, for either display type. However, brief periods of adaptation could induce pop-out responses in V2. This suggests that adaptation might define which features of the environment are most salient, even if those features would otherwise not evoke pop-out responses. ### Competing Interest Statement The authors have declared no competing interest.

Modern recording techniques now allow us to record from distinct neuronal populations in different brain networks. However, especially as we consider multiple (more than two) populations, new conceptual and statistical frameworks are needed to characterize the multi-dimensional, concurrent flow of signals among these populations. Here, we develop a dimensionality reduction framework that determines (1) the subset of populations described by each latent dimension, (2) the direction of signal flow among those populations, and (3) how those signals evolve over time within and across experimental trials. We illustrate these features in simulation, and further validate the method by applying it to previously studied recordings from neuronal populations in macaque visual areas V1 and V2. Then we study interactions across select laminar compartments of areas V1, V2, and V3d, recorded simultaneously with multiple Neuropixels probes. Our approach uncovered signatures of selective communication across these three areas that related to their retinotopic alignment. This work advances the study of concurrent signaling across multiple neuronal populations.

Abstract Nearly all brain functions involve activity distributed across multiples areas or structures. Function requires that activity is appropriately routed and coordinated across the areas relevant for a particular task. Anatomy informs what communication is possible, but the dynamic and flexible nature of inter-areal signaling indicates that there are mechanisms that can modulate communication on a moment-by-moment basis. This chapter reviews current understanding of these mechanisms, and discusses several open issues.

Brain function relies on the coordination of activity across multiple, recurrently connected brain areas. For instance, sensory information encoded in early sensory areas is relayed to, and further processed by, higher cortical areas and then fed back. However, the way in which feedforward and feedback signaling interact with one another is incompletely understood. Here we investigate this question by leveraging simultaneous neuronal population recordings in early and midlevel visual areas (V1–V2 and V1–V4). Using a dimensionality reduction approach, we find that population interactions are feedforward-dominated shortly after stimulus onset and feedback-dominated during spontaneous activity. The population activity patterns most correlated across areas were distinct during feedforward- and feedback-dominated periods. These results suggest that feedforward and feedback signaling rely on separate “channels”, which allows feedback signals to not directly affect activity that is fed forward. How cortical areas interact via feedforward and feedback signaling remains unclear. Here, the authors recorded from V1 and V2/V4 in macaque visual cortex and found that feedforward and feedback interactions vary with stimulus drive and involve different neuronal population activity patterns.

Technological advances now allow us to record from large populations of neurons across multiple brain areas. These recordings may illuminate how communication between areas contributes to brain function, yet a substantial barrier remains: how do we disentangle the concurrent, bidirectional flow of signals between populations of neurons? We propose here a dimensionality reduction framework, delayed latents across groups (DLAG), that disentangles signals relayed in each direction, identifies how these signals are represented by each population and characterizes how they evolve within and across trials. We demonstrate that DLAG performs well on synthetic datasets similar in scale to current neurophysiological recordings. Then we study simultaneously recorded populations in primate visual areas V1 and V2, where DLAG reveals signatures of bidirectional yet selective communication. Our framework lays a foundation for dissecting the intricate flow of signals across populations of neurons, and how this signalling contributes to cortical computation.

Visual perception depends strongly on spatial context. A profound example is visual crowding, whereby the presence of nearby stimuli impairs the discriminability of object features. Despite extensive work on percep-tual crowding and the spatial integrative properties of visual cortical neurons, the link between these two as-pects of visual processing remains unclear. To understand better the neural basis of crowding, we recorded activity simultaneously from neuronal populations in V1 and V4 of fixating macaque monkeys. We assessed the information available from the measured responses about the orientation of a visual target both for tar-gets presented in isolation and amid distractors. Both single neuron and population responses had less in-formation about target orientation when distractors were present. Information loss was moderate in V1 and more substantial in V4. Information loss could be traced to systematic divisive and additive changes in neuronal tuning. Additive and multiplicative changes in tuning were more severe in V4; in addition, tuning ex-hibited other, non-affine transformations that were greater in V4, further restricting the ability of a fixed sen-sory readout strategy to extract accurate feature information across displays. Our results provide a direct test of crowding effects at different stages of the visual hierarchy. They reveal how crowded visual environments alter the spiking activity of cortical populations by which sensory stimuli are encoded and connect these changes to established mechanisms of neuronal spatial integration.

A recent formulation of predictive coding theory proposes that a subset of neurons in each cortical area encodes sensory prediction errors, the difference between predictions relayed from higher cortex and the sensory input. Here, we test for evidence of prediction error responses in spiking responses and local field potentials (LFP) recorded in primary visual cortex and area V4 of macaque monkeys, and in complementary electroencephalographic (EEG) scalp recordings in human participants. We presented a fixed sequence of visual stimuli on most trials, and violated the expected ordering on a small subset of trials. Under predictive coding theory, pattern-violating stimuli should trigger robust prediction errors, but we found that spiking, LFP and EEG responses to expected and pattern-violating stimuli were nearly identical. Our results challenge the assertion that a fundamental computational motif in sensory cortex is to signal prediction errors, at least those based on predictions derived from temporal patterns of visual stimulation.

This data is associated to the following paper: Festa D., Aschner A, Davila A, Kohn A, Coen-Cagli R. Neuronal variability reflects probabilistic inference tuned to natural image statistics. The data consists of: 1) photographic natural images from the BSD500 dataset https://github.com/BIDS/BSDS, used to train the Gaussian Scale Mixture model. Model equations and implementation details are fully described in the associated paper. 2) multi-electrode recordings from V1 in anesthetized and awake macaque monkeys, while natural images and gratings were flashed on the screen. Recordings were performed using “Utah” electrode arrays. Images were presented at different sizes and orientations, to quantify surround modulation of response strength and variability in single neurons. Experimental procedures and stimuli are fully described in the associated paper. Code to read in and process this dataset is provided at https://github.com/rubencoencagli/festa-et-al-2020 . The code reproduces the main figures of the associated paper.

Neuronal activity in sensory cortex fluctuates over time and across repetitions of the same input. This variability is often considered detrimental to neural coding. The theory of neural sampling proposes instead that variability encodes the uncertainty of perceptual inferences. In primary visual cortex (V1), modulation of variability by sensory and non-sensory factors supports this view. However, it is unknown whether V1 variability reflects the statistical structure of visual inputs, as would be required for inferences correctly tuned to the statistics of the natural environment. Here we combine analysis of image statistics and recordings in macaque V1 to show that probabilistic inference tuned to natural image statistics explains the widely observed dependence between spike-count variance and mean, and the modulation of V1 activity and variability by spatial context in images. Our results show that the properties of a basic aspect of cortical responses — their variability — can be explained by a probabilistic representation tuned to naturalistic inputs.

The neural basis of perceptual decision making has typically been studied using measurements of single neuron activity, though decisions are likely based on the activity of large neuronal ensembles. Local field potentials (LFPs) may, in some cases, serve as a useful proxy for population activity and thus be useful for understanding the neural basis of perceptual decision making. However, little is known about whether LFPs in sensory areas include decision-related signals. We therefore analyzed LFPs recorded using two 48-electrode arrays implanted in primary visual cortex (V1) and area V4 of macaque monkeys trained to perform a fine orientation discrimination task. We found significant choice information in low (0-30 Hz) and higher (70-500 Hz) frequency components of the LFP, but little information in gamma frequencies (30-70 Hz). Choice information was more robust in V4 than V1 and stronger in LFPs than in simultaneously measured spiking activity. LFP-based choice information included a global component, common across electrodes within an area. Our findings reveal the presence of robust choice-related signals in the LFPs recorded in V1 and V4 and suggest that LFPs may be a useful complement to spike-based analyses of decision making.

Optogenetics has revolutionized neuroscience in small laboratory animals, but its effect on animal models more closely related to humans, such as non-human primates (NHPs), has been mixed. To make evidence-based decisions in primate optogenetics, the scientific community would benefit from a centralized database listing all attempts, successful and unsuccessful, of using optogenetics in the primate brain. We contacted members of the community to ask for their contributions to an open science initiative. As of this writing, 45 laboratories around the world contributed more than 1,000 injection experiments, including precise details regarding their methods and outcomes. Of those entries, more than half had not been published. The resource is free for everyone to consult and contribute to on the Open Science Framework website. Here we review some of the insights from this initial release of the database and discuss methodological considerations to improve the success of optogenetic experiments in NHPs.

Texture synthesis models are important tools for understanding visual processing. In particular, statistical approaches based on neurally relevant features have been instrumental in understanding aspects of visual perception and of neural coding. New deep learning-based approaches further improve the quality of synthetic textures. Yet, it is still unclear why deep texture synthesis performs so well, and applications of this new framework to probe visual perception are scarce. Here, we show that distributions of deep convolutional neural network (CNN) activations of a texture are well described by elliptical distributions and therefore, following optimal transport theory, constraining their mean and covariance is sufficient to generate new texture samples. Then, we propose the natural geodesics (ie the shortest path between two points) arising with the optimal transport metric to interpolate between arbitrary textures. Compared to other CNN-based approaches, our interpolation method appears to match more closely the geometry of texture perception, and our mathematical framework is better suited to study its statistical nature. We apply our method by measuring the perceptual scale associated to the interpolation parameter in human observers, and the neural sensitivity of different areas of visual cortex in macaque monkeys.