The classic view that neural populations in sensory cortices preferentially encode responses to incoming stimuli has been strongly challenged by recent experimental studies. Despite the fact that a large fraction of variance of visual responses in rodents can be attributed to behavioral state and movements, trial-history, and salience, the effects of contextual modulations and expectations on sensory-evoked responses in visual and association areas remain elusive. Here, we present a comprehensive experimental and theoretical study showing that hierarchically connected visual and association areas differentially encode the temporal context and expectation of naturalistic visual stimuli, consistent with the theory of hierarchical predictive coding. We measured neural responses to expected and unexpected sequences of natural scenes in the primary visual cortex (V1), the posterior medial higher order visual area (PM), and retrosplenial cortex (RSP) using 2-photon imaging in behaving mice collected through the Allen Institute Mindscope's OpenScope program. We found that information about image identity in neural population activity depended on the temporal context of transitions preceding each scene, and decreased along the hierarchy. Furthermore, our analyses revealed that the conjunctive encoding of temporal context and image identity was modulated by expectations of sequential events. In V1 and PM, we found enhanced and specific responses to unexpected oddball images, signaling stimulus-specific expectation violation. In contrast, in RSP the population response to oddball presentation recapitulated the missing expected image rather than the oddball image. These differential responses along the hierarchy are consistent with classic theories of hierarchical predictive coding whereby higher areas encode predictions and lower areas encode deviations from expectation. We further found evidence for drift in visual responses on the timescale of minutes. Although activity drift was present in all areas, population responses in V1 and PM, but not in RSP, maintained stable encoding of visual information and representational geometry. Instead we found that RSP drift was independent of stimulus information, suggesting a role in generating an internal model of the environment in the temporal domain. Overall, our results establish temporal context and expectation as substantial encoding dimensions in the visual cortex subject to fast representational drift and suggest that hierarchically connected areas instantiate a predictive coding mechanism.

Multiple recent studies have shown that motor activity greatly impacts the activity of primary sensory areas like V1. Yet, the role of this motor related activity in sensory processing is still unclear. Here, we dissect how these behavior signals are broadcast to different layers and areas of the visual cortex. To do so, we leveraged a standardized and spontaneous behavioral fidget event in passively viewing mice. Importantly, this behavior event had no relevance to any ongoing task allowing us to compare its neuronal correlates with visually relevant behaviors (e.g., running). A large two-photon Ca2+ imaging database of neuronal responses uncovered four neural response types during fidgets that were consistent in their proportion and response patterns across all visual areas and layers of the visual cortex. Indeed, the layer and area identity could not be decoded above chance level based only on neuronal recordings. In contrast to running behavior, fidget evoked neural responses that were independent to visual processing. The broad availability of visually orthogonal standardized behavior signals could be a key component in how the cortex selects, learns and binds local sensory information with motor outputs. Contrary to behaviorally relevant motor outputs, irrelevant motor signals could project to separate local neural subspaces.

The organization and properties of the neural receptive fields in the visual cortex is a prominent factor in understanding the mechanism of neural coding. Here, we perform a systematic and standardized analysis of the structure, organization and properties of the receptive fields in the pan-excitatory neuronal population in the mouse primary visual cortex (area V1). We collected and analyzed a large-scale 2-photon volumetric imaging dataset from an 800x800x600 μm^3 volume in V1 of two mice. The visual responses from were recorded during presentation of a locally sparse noise stimulus to efficiently reconstruct the neural receptive fields. The locally sparse noise stimulus consisted of 9 degree black or white spots on a mean luminance gray background presented at ~3Hz. We designed and implemented a computational pipeline to identify responsive trials, ON and OFF subfields and their geometric properties. Using these experimental and computational pipelines, we found that OFF subfields tended to have larger areas than ON subfields, particularly in layer 2/3. On average, OFF subfields were 15.6 degree^2 larger than the ON subfields across all imaging depths. We also found that the average receptive field area decreased as a function of cortical depth. ON and OFF subfields in layer 5 were on average 22% (ON subfields) and 31.5% (OFF subfields) smaller than those in layer 2/3. Additionally, we found that the responsiveness rate to the sparse noise decreased in deeper layers of cortex. Neurons in layer 2/3 were about 4 times more likely to respond to our visual stimulus compared to neurons in layer 5. Our results provide a systematic and standardized survey of the receptive field properties in V1. The dataset and computational pipeline could serve as a valuable resource to the community to further study the visual responses of neurons in V1.

Vasoactive intestinal peptide-expressing (VIP) interneurons in the cortex regulate feedback inhibition of pyramidal neurons through suppression of somatostatin-expressing (SST) interneurons and, reciprocally, SST neurons inhibit VIP neurons. Although VIP neuron activity in the primary visual cortex (V1) of mouse is highly correlated with locomotion, the relevance of locomotion-related VIP neuron activity to visual coding is not known. Here we show that VIP neurons in mouse V1 respond strongly to low contrast front-to-back motion that is congruent with self-motion during locomotion but are suppressed by other directions and contrasts. VIP and SST neurons have complementary contrast tuning. Layer 2/3 contains a substantially larger population of low contrast preferring pyramidal neurons than deeper layers, and layer 2/3 (but not deeper layer) pyramidal neurons show bias for front-to-back motion specifically at low contrast. Network modeling indicates that VIP-SST mutual antagonism regulates the gain of the cortex to achieve sensitivity to specific weak stimuli without compromising network stability.

Multiple recent studies have shown that motor activity greatly impacts the activity of primary sensory areas like V1. Yet, the role of this motor related activity in sensory processing is still unclear. Here we further dissect how these behavior relevant signals are broadcast to different layers and areas of visual cortex. To do so, we leveraged a standardized motor behavior fidget event in behavioral videos of passively viewing mice. A large two-photon Ca2+ imaging database of neuronal responses uncovered four neural response types during fidgets that are surprisingly consistent in their proportion and response patterns across all visual areas and layers of the visual cortex. Indeed, the layer and area identity could not be decoded above chance level based only on neuronal recordings. The broad availability of standardized behavior signals could be a key component in how the cortex selects, learns and binds local sensory information with relevant motor outputs.

Direction selectivity – the preference of motion in one direction over the opposite – is a fundamental property of visual neurons across species. We find that a substantial proportion of direction selective neurons in the mouse visual system reverse their preferred direction of motion in response to drifting gratings at different spatiotemporal parameters. A spatiotemporally asymmetric filter model recapitulates our experimental observations.

To understand how the brain processes sensory information to guide behavior, we must know how stimulus representations are transformed throughout the visual cortex. Here we report an open, large-scale physiological survey of activity in the awake mouse visual cortex: the Allen Brain Observatory Visual Coding dataset. This publicly available dataset includes the cortical activity of nearly 60,000 neurons from six visual areas, four layers, and 12 transgenic mouse lines in a total of 243 adult mice, in response to a systematic set of visual stimuli. We classify neurons on the basis of joint reliabilities to multiple stimuli and validate this functional classification with models of visual responses. While most classes are characterized by responses to specific subsets of the stimuli, the largest class is not reliably responsive to any of the stimuli and becomes progressively larger in higher visual areas. These classes reveal a functional organization wherein putative dorsal areas show specialization for visual motion signals.

Vasoactive intestinal peptide-expressing (VIP) interneurons in cortex regulate feedback inhibition of pyramidal neurons through suppression of somatostatin-expressing (SST) interneurons and, reciprocally, SST neurons inhibit VIP neurons. Here, we show that VIP neurons in mouse primary visual cortex have complementary contrast tuning to SST neurons and respond synergistically to front-to-back visual motion and locomotion. Network modeling indicates that this VIP-SST mutual antagonism regulates the gain of cortex to achieve both sensitivity to behaviorally-relevant stimuli and network stability.

2-photon image quality degrades in volume-labeled tissue due to high background fluorescence. We apply 3-photon imaging to measuring functional responses in deep layers of primary visual cortex in pan-excitatory transgenic mice expressing GCaMP6s.

We report a novel two-photon fluorescence microscope based on a fast-switching liquid crystal spatial light modulator and a pair of galvo-resonant scanners for large-scale recording of neural activity from the mammalian brain. The utilized imaging technique is capable of monitoring large populations of neurons spread across different layers of the neocortex in awake and behaving mice. During each imaging session, all visual stimulus driven somatic activity could be recorded in the same behavior state. We observed heterogeneous response to different types of visual stimuli from ~ 3,300 excitatory neurons reaching from layer II/III to V of the striate cortex.