In 1949, Donald Hebb proposed that neuronal assemblies with temporally specific patterns of activity form the building blocks of perception, cognition, and behavior. Finding the structural underpinning of such assemblies has been technically challenging due to a lack of large-scale structure-activity maps. Here, we combine in vivo optical physiology with postmortem electron microscopy (EM) in the same tissue volume. Using higher-order correlations in fluorescence traces, we extract neuronal assemblies. Physiologically, we show that these assemblies respond more reliably to repeated natural movies than size-matched control ensembles and decode such stimuli more accurately. Structurally, we find that over a quarter of the pyramidal neurons do not participate in any assembly and are significantly less integrated into the connectome than those that do. We do not observe a marked increase in the strength of monosynaptic excitatory connections between neurons sharing assembly assignment, but instead find significantly stronger indirect inhibitory connections targeting cells in other assemblies. These results show that assemblies can serve as functional units of perception and suggest they may be structurally delineated by mutual inhibition.

Neural circuit function is shaped both by the cell types that comprise the circuit and the connections between them1. Neural cell types have previously been defined by morphology2,3, electrophysiology4, transcriptomic expression5,6, connectivity7-9 or a combination of such modalities10-12. The Patch-seq technique enables the characterization of morphology, electrophysiology and transcriptomic properties from individual cells13-15. These properties were integrated to define 28 inhibitory, morpho-electric-transcriptomic (MET) cell types in mouse visual cortex16, which do not include synaptic connectivity. Conversely, large-scale electron microscopy (EM) enables morphological reconstruction and a near-complete description of a neuron's local synaptic connectivity, but does not include transcriptomic or electrophysiological information. Here, we leveraged morphological information from Patch-seq to predict the transcriptomically defined cell subclass and/or MET-type of inhibitory neurons within a large-scale EM dataset. We further analysed Martinotti cells-a somatostatin (Sst)-positive17 morphological cell type18,19-which were classified successfully into Sst MET-types with distinct axon myelination and synaptic output connectivity patterns. We demonstrate that morphological features can be used to link cell types across experimental modalities, enabling further comparison of connectivity to gene expression and electrophysiology. We observe unique connectivity rules for predicted Sst cell types.

Despite significant progress in characterizing neocortical cell types, a complete understanding of the synaptic connections of individual excitatory cells remains elusive. This study investigates the connectivity of mouse visual cortex thick tufted layer 5 pyramidal cells, also known as extratelencephalic neurons (L5-ETns), using a 1 mm3 publicly available electron microscopy dataset. The analysis reveals that, in their immediate vicinity, L5-ETns primarily establish connections with a group of inhibitory cell types, which, in turn, specifically target the L5-ETns back. The most common excitatory targets of L5-ETns are layer 5 intertelencephalic neurons (L5-ITns) and layer 6 (L6) pyramidal cells, whereas synapses with other L5-ETns are less common. When L5-ETns extend their axons to other cortical regions, they tend to connect more with excitatory cells. Our results highlight a circuit motif where a subclass of excitatory cells forms a subcircuit with specific inhibitory cell types. This is achieved using a publicly available, automated approach for synapse recognition and automated cell typing, offering a framework for exploring the connectivity of other neuron types.

Advances in Electron Microscopy, image segmentation and computational infrastructure have given rise to large-scale and richly annotated connectomic datasets which are increasingly shared across communities. To enable collaboration, users need to be able to concurrently create new annotations and correct errors in the automated segmentation by proofreading. In large datasets, every proofreading edit relabels cell identities of millions of voxels and thousands of annotations like synapses. For analysis, users require immediate and reproducible access to this constantly changing and expanding data landscape. Here, we present the Connectome Annotation Versioning Engine (CAVE), a computational infrastructure for immediate and reproducible connectome analysis in up-to petascale datasets (~1mm3) while proofreading and annotating is ongoing. For segmentation, CAVE provides a distributed proofreading infrastructure for continuous versioning of large reconstructions. Annotations in CAVE are defined by locations such that they can be quickly assigned to the underlying segment which enables fast analysis queries of CAVE's data for arbitrary time points. CAVE supports schematized, extensible annotations, so that researchers can readily design novel annotation types. CAVE is already used for many connectomics datasets, including the largest datasets available to date.

Mammalian cortex features a vast diversity of neuronal cell types, each with characteristic anatomical, molecular and functional properties1. Synaptic connectivity shapes how each cell type participates in the cortical circuit, but mapping connectivity rules at the resolution of distinct cell types remains difficult. Here we used millimetre-scale volumetric electron microscopy2 to investigate the connectivity of all inhibitory neurons across a densely segmented neuronal population of 1,352 cells spanning all layers of mouse visual cortex, producing a wiring diagram of inhibition with more than 70,000 synapses. Inspired by classical neuroanatomy, we classified inhibitory neurons based on targeting of dendritic compartments and developed an excitatory neuron classification based on dendritic reconstructions with whole-cell maps of synaptic input. Single-cell connectivity showed a class of disinhibitory specialist that targets basket cells. Analysis of inhibitory connectivity onto excitatory neurons found widespread specificity, with many interneurons exhibiting differential targeting of spatially intermingled subpopulations. Inhibitory targeting was organized into 'motif groups', diverse sets of cells that collectively target both perisomatic and dendritic compartments of the same excitatory targets. Collectively, our analysis identified new organizing principles for cortical inhibition and will serve as a foundation for linking contemporary multimodal neuronal atlases with the cortical wiring diagram.

Mammalian cortex features a vast diversity of neuronal cell types, each with characteristic anatomical, molecular and functional properties. Synaptic connectivity powerfully shapes how each cell type participates in the cortical circuit, but mapping connectivity rules at the resolution of distinct cell types remains difficult. Here, we used millimeter-scale volumetric electron microscopy1to investigate the connectivity of all inhibitory neurons across a densely-segmented neuronal population of 1352 cells spanning all layers of mouse visual cortex, producing a wiring diagram of inhibitory connections with more than 70,000 synapses. Taking a data-driven approach inspired by classical neuroanatomy, we classified inhibitory neurons based on the relative targeting of dendritic compartments and other inhibitory cells and developed a novel classification of excitatory neurons based on the morphological and synaptic input properties. The synaptic connectivity between inhibitory cells revealed a novel class of disinhibitory specialist targeting basket cells, in addition to familiar subclasses. Analysis of the inhibitory connectivity onto excitatory neurons found widespread specificity, with many interneurons exhibiting differential targeting of certain subpopulations spatially intermingled with other potential targets. Inhibitory targeting was organized into “motif groups,” diverse sets of cells that collectively target both perisomatic and dendritic compartments of the same excitatory targets. Collectively, our analysis identified new organizing principles for cortical inhibition and will serve as a foundation for linking modern multimodal neuronal atlases with the cortical wiring diagram.

The detection of novel stimuli is critical to learn and survive in a dynamic environment. Though novel stimuli powerfully affect brain activity, their impact on specific cell types and circuits is not well understood. Disinhibition is one candidate mechanism for novelty-induced enhancements in activity. Here we characterize the impact of stimulus novelty on disinhibitory circuit components using longitudinal 2-photon calcium imaging of Vip, Sst, and excitatory populations in the mouse visual cortex. Mice learn a behavioral task with stimuli that become highly familiar, then are tested on both familiar and novel stimuli. Mice consistently perform the task with novel stimuli, yet responses to stimulus presentations and stimulus omissions are dramatically altered. Further, we find that novelty modifies coding of visual as well as behavioral and task information. At the population level, the direction of these changes is consistent with engagement of the Vip-Sst disinhibitory circuit. At the single cell level, we identify separate clusters of Vip, Sst, and excitatory cells with unique patterns of novelty-induced coding changes. This study and the accompanying open-access dataset reveals the impact of novelty on sensory and behavioral representations in visual cortical circuits and establishes novelty as a key driver of cellular functional diversity.

Serial-section electron microscopy (ssEM) is the method of choice for studying macroscopic biological samples at extremely high resolution in three dimensions. In the nervous system, nanometer-scale images are necessary to reconstruct dense neural wiring diagrams in the brain, so-called connectomes. The data that can comprise of up to 10(8) individual EM images must be assembled into a volume, requiring seamless 2D registration from physical section followed by 3D alignment of the stitched sections. The high throughput of ssEM necessitates 2D stitching to be done at the pace of imaging, which currently produces tens of terabytes per day. To achieve this, we present a modular volume assembly software pipeline ASAP (Assembly Stitching and Alignment Pipeline) that is scalable to datasets containing petabytes of data and parallelized to work in a distributed computational environment. The pipeline is built on top of the Render Trautman and Saalfeld (2019) services used in the volume assembly of the brain of adult Drosophila melanogaster (Zheng et al. 2018). It achieves high throughput by operating only on image meta-data and transformations. ASAP is modular, allowing for easy incorporation of new algorithms without significant changes in the workflow. The entire software pipeline includes a complete set of tools for stitching, automated quality control, 3D section alignment, and final rendering of the assembled volume to disk. ASAP has been deployed for continuous stitching of several large-scale datasets of the mouse visual cortex and human brain samples including one cubic millimeter of mouse visual cortex (Yin et al. 2020); Microns Consortium et al. (2021) at speeds that exceed imaging. The pipeline also has multi-channel processing capabilities and can be applied to fluorescence and multi-modal datasets like array tomography.

To understand the brain we must relate neurons’ functional responses to the circuit architecture that shapes them. Here, we present a large functional connectomics dataset with dense calcium imaging of a millimeter scale volume. We recorded activity from approximately 75,000 neurons in primary visual cortex (VISp) and three higher visual areas (VISrl, VISal and VISlm) in an awake mouse viewing natural movies and synthetic stimuli. The functional data were co-registered with a volumetric electron microscopy (EM) reconstruction containing more than 200,000 cells and 0.5 billion synapses. Subsequent proofreading of a subset of neurons in this volume yielded reconstructions that include complete dendritic trees as well the local and inter-areal axonal projections that map up to thousands of cell-to-cell connections per neuron. Here, we release this dataset as an open-access resource to the scientific community including a set of tools that facilitate data retrieval and downstream analysis. In accompanying papers we describe our findings using the dataset to provide a comprehensive structural characterization of cortical cell types 1–3 and the most detailed synaptic level connectivity diagram of a cortical column to date 2 , uncovering unique cell-type specific inhibitory motifs that can be linked to gene expression data 4 . Functionally, we identify new computational principles of how information is integrated across visual space 5 , characterize novel types of neuronal invariances 6 and bring structure and function together to decipher a general principle that wires excitatory neurons within and across areas 7, 8 .

3D electron microscopy (EM) has been successful at mapping invertebrate nervous systems, but the approach has been limited to small chunks of mammalian brains. To scale up to larger volumes, we have built a computational pipeline for processing petascale image datasets acquired by serial section EM, a popular form of 3D EM. The pipeline employs convolutional nets to compute the nonsmooth transformations required to align images of serial sections containing numerous cracks and folds, detect neuronal boundaries, label voxels as axon, dendrite, soma, and other semantic categories, and detect synapses and assign them to presynaptic and postsynaptic segments. The output of neuronal boundary detection is segmented by mean affinity agglomeration with semantic and size constraints. Pipeline operations are implemented by leveraging distributed and cloud computing. Intermediate results of the pipeline are held in cloud storage, and can be effortlessly viewed as images, which aids debugging. We applied the pipeline to create an automated reconstruction of an EM image volume spanning four visual cortical areas of a mouse brain. Code for the pipeline is publicly available, as is the reconstructed volume.

Morphology based analysis of cell types has been an area of great interest to the neuroscience community for several decades. Recently, high resolution electron microscopy (EM) datasets of the mouse brain have opened up opportunities for data analysis at a level of detail that was previously impossible. These datasets are very large in nature and thus, manual analysis is not a practical solution. Of particular interest are details to the level of post synaptic structures. This paper proposes a fully automated framework for analysis of post-synaptic structure based neuron analysis from EM data. The processing framework involves shape extraction, representation with an autoencoder, and whole cell modeling and analysis based on shape distributions. We apply our novel framework on a dataset of 1031 neurons obtained from imaging a 1mm x 1mm x 40 micrometer volume of the mouse visual cortex and show the strength of our method in clustering and classification of neuronal shapes.

Electron microscopy (EM) is widely used for studying cellular structure and network connectivity in the brain. We have built a parallel imaging pipeline using transmission electron microscopes that scales this technology, implements 24/7 continuous autonomous imaging, and enables the acquisition of petascale datasets. The suitability of this architecture for large-scale imaging was demonstrated by acquiring a volume of more than 1 mm 3 of mouse neocortex, spanning four different visual areas at synaptic resolution, in less than 6 months. Over 26,500 ultrathin tissue sections from the same block were imaged, yielding a dataset of more than 2 petabytes. The combined burst acquisition rate of the pipeline is 3 Gpixel per sec and the net rate is 600 Mpixel per sec with six microscopes running in parallel. This work demonstrates the feasibility of acquiring EM datasets at the scale of cortical microcircuits in multiple brain regions and species.