Understanding the brain increasingly depends on integrating evidence across scales, modalities, and disciplines. Addressing a single research question therefore requires a coordinated sequence of operations, from surveying prior work to executing analyses and interpreting results in light of domain knowledge. AI agents promise to accelerate this process, but current agents lack domain expertise in brain science, may fabricate claims, drift during multi-step reasoning, and offer few defined points for expert intervention. These failures are especially costly in brain science, where conclusions feed into downstream scientific claims and depend on laboratory-specific expertise and careful human judgment. We present BrainPilot a fully open-source multi-agent system that accelerates brain science research with traceable logs and agent-verified results. A principal investigator (PI) agent coordinates specialist agents grounded in curated domain knowledge: a unified brain science knowledge base containing 7,233 indexed items and a skill library of 72 reusable methodology units across seven research domains. Every major step is recorded in the Graph of Trace, an auditable record that links subgoals, tool use, evidence, and claims and allows researchers to follow and inspect the workflow. An Auditor agent further integrates fabrication checking into the workflow. For evaluation, we run three brain science tasks from Agents' Last Exam, introduce our own benchmark, BrainPilotBench-v0, and present additional end-to-end case studies. Across these evaluations, BrainPilot with an open-source backbone model attains performance comparable to state-of-the-art agent framework with less costs.

The brain contains an astronomical number of neurons, but it is their collective activity that underlies brain function. The number of degrees of freedom that this collective activity explores – its dimensionality – is therefore a fundamental signature of neural dynamics and computation (1–7). However, it is not known what controls this dimensionality in the biological brain – and in particular whether and how recurrent synaptic networks play a role (8–10). Through analysis of high-density Neuropixels recordings (11), we argue that areas across the mouse cortex operate in asensitive regimethat gives these synaptic networks a very strong role in controlling dimensionality. We show that this control is expressed across time, as cortical activity transitions among states with different dimensionalities. Moreover, we show that the control is mediated through highly tractable features of synaptic networks. We then analyze these key features via a massive synaptic physiology dataset (12). Quantifying these features in terms of cell-type specific network motifs, we find that the synaptic patterns that impact dimensionality are prevalent in both mouse and human brains. Thus local circuitry scales up systematically to help control the degrees of freedom that brain networks may explore and exploit.

In a recent study, Ding, Tran, and colleagues applied model-guided closed-loop physiology to systematically map neuronal selectivity and invariance in the mouse primary visual cortex, revealing a spectrum of receptive-field invariances, including a bipartite organization tuned to spatial-frequency boundaries. This approach efficiently explores the high-dimensional visual feature space, providing a framework to investigate neuronal coding schemes and representational geometry.

Abstract The brain can rapidly adjust sensory processing according to behavioral context from moment to moment without altering its underlying anatomical wiring. Such flexibility is thought to arise from dynamic reconfiguration of the effective network through which signals propagate, yet the principles governing such reconfiguration and its consequences for coding remain unclear. Here, we exploited the distinct stationary and locomotion states within the same recording sessions and addressed this question by inferring directed, millisecond-scale signal-transmission networks at single-neuron resolution in behaving mice. Locomotion was accompanied by a counterintuitive structured sparsification of the inferred network: interactions became fewer but more temporally precise, while the remaining interactions were more local, modular, feature-specific, and feedforward. We then used theoretical analysis and controlled perturbations of multi-area rate models to systematically determine how each empirically observed component of this reorganization affects population coding. We found that sparsification reduced shared variability; local organization reduced signal–noise alignment; feature-specific interactions sharpened selectivity; and a more feedforward architecture accelerated decoding. These results provide an experimentally grounded mechanism by which behavioral state reorganizes neuronal interactions given the same sensory inputs, and suggest structured sparsification as an underlying principle of network reconfiguration that supports accurate and faster sensory coding.

Influenced by non-stationary factors such as brain states and behavior, neurons exhibit substantial response variability even to identical stimuli. However, it remains unclear how their relative impact on neuronal variability evolves over time. To address this question, we designed an encoding model conditioned on latent states to partition variability in the mouse visual cortex across internal brain dynamics, behavior, and external visual stimulus. Applying a hidden Markov model to local field potentials, we consistently identified three distinct oscillation states, each with a unique variability profile. Regression models within each state revealed a dynamic composition of factors influencing spiking variability, with the dominant factor switching within seconds. The state-conditioned regression model uncovered extensive diversity in source contributions across units, varying in accordance with anatomical hierarchy and internal state. This heterogeneity in encoding underscores the importance of partitioning variability over time, particularly when considering the influence of non-stationary factors on sensory processing.

To understand the neural basis of behavior, it is essential to sensitively and accurately measure neural activity at single-neuron and single-spike resolution. Extracellular electrophysiology delivers this, but it has biases in the neurons it detects and it imperfectly resolves their action potentials. To minimize these limitations, we developed a silicon probe with much smaller and denser recording sites than previous designs, called Neuropixels Ultra (NP Ultra). Using NP Ultra, neuronal yield in mouse visual cortex recordings increased by more than 2-fold. With ultra-high spatial resolution, we discovered that a feature of extracellular waveforms, the spatial extent or “footprint,” distinguished axonal from somatic recordings. In addition, three genetically identified cortical cell types could be discriminated from one another with ∼80% accuracy and from other neurons with ∼85% accuracy. NP Ultra improves yield, detection of subcellular compartments, and cell type identification to enable a more powerful dissection of neural circuit activity during behavior.

To understand the neural basis of behavior, it is essential to sensitively and accurately measure neural activity at single neuron and single spike resolution. Extracellular electrophysiology delivers this, but it has biases in the neurons it detects and it imperfectly resolves their action potentials. To minimize these limitations, we developed a silicon probe with much smaller and denser recording sites than previous designs, called Neuropixels Ultra (NP Ultra). This device samples neuronal activity at ultra-high spatial density (~10 times higher than previous probes) with low noise levels, while trading off recording span. NP Ultra is effectively an implantable voltage-sensing camera that captures a planar image of a neuron's electrical field. We use a spike sorting algorithm optimized for these probes to demonstrate that the yield of visually-responsive neurons in recordings from mouse visual cortex improves up to ~3-fold. We show that NP Ultra can record from small neuronal structures including axons and dendrites. Recordings across multiple brain regions and four species revealed a subset of extracellular action potentials with unexpectedly small spatial spread and axon-like features. We share a large-scale dataset of these brain-wide recordings in mice as a resource for studies of neuronal biophysics. Finally, using ground-truth identification of three major inhibitory cortical cell types, we found that these cell types were discriminable with approximately 75% success, a significant improvement over lower-resolution recordings. NP Ultra improves spike sorting performance, detection of subcellular compartments, and cell type classification to enable more powerful dissection of neural circuit activity during behavior.

On the timescale of sensory processing, neuronal networks have relatively fixed anatomical connectivity, while functional interactions between neurons can vary depending on the ongoing activity of the neurons within the network. We thus hypothesized that different types of stimuli could lead those networks to display stimulus-dependent functional connectivity patterns. To test this hypothesis, we analyzed single-cell resolution electrophysiological data from the Allen Institute, with simultaneous recordings of stimulus-evoked activity from neurons across 6 different regions of mouse visual cortex. Comparing the functional connectivity patterns during different stimulus types, we made several nontrivial observations: (1) while the frequencies of different functional motifs were preserved across stimuli, the identities of the neurons within those motifs changed; (2) the degree to which functional modules are contained within a single brain region increases with stimulus complexity. Altogether, our work reveals unexpected stimulus-dependence to the way groups of neurons interact to process incoming sensory information.

The mammalian cerebral cortex is anatomically organized into a six-layer motif. It is currently unknown whether a corresponding laminar motif of neuronal activity patterns exists across the cortex. Here we report such a motif in the power of local field potentials (LFPs). Using laminar probes, we recorded LFPs from 14 cortical areas across the cortical hierarchy in five macaque monkeys. The laminar locations of recordings were histologically identified by electrolytic lesions. Across all areas, we found a ubiquitous spectrolaminar pattern characterized by an increasing deep-to-superficial layer gradient of high-frequency power peaking in layers 2/3 and an increasing superficial-to-deep gradient of alpha-beta power peaking in layers 5/6. Laminar recordings from additional species showed that the spectrolaminar pattern is highly preserved among primates—macaque, marmoset and human—but more dissimilar in mouse. Our results suggest the existence of a canonical layer-based and frequency-based mechanism for cortical computation.

The brain consists of many cell classes yet in vivo electrophysiology recordings are typically unable to identify and monitor their activity in the behaving animal. Here, we employed a systematic approach to link cellular, multi-modal in vitro properties from experiments with in vivo recorded units via computational modeling and optotagging experiments. We found two one-channel and six multi-channel clusters in mouse visual cortex with distinct in vivo properties in terms of activity, cortical depth, and behavior. We used biophysical models to map the two one- and the six multi-channel clusters to specific in vitro classes with unique morphology, excitability and conductance properties that explain their distinct extracellular signatures and functional characteristics. These concepts were tested in ground-truth optotagging experiments with two inhibitory classes unveiling distinct in vivo properties. This multi-modal approach presents a powerful way to separate in vivo clusters and infer their cellular properties from first principles.

The detection of novel stimuli is critical to learn and survive in a dynamic environment. Though novel stimuli powerfully affect brain activity, their impact on specific cell types and circuits is not well understood. Disinhibition is one candidate mechanism for novelty-induced enhancements in activity. Here we characterize the impact of stimulus novelty on disinhibitory circuit components using longitudinal 2-photon calcium imaging of Vip, Sst, and excitatory populations in the mouse visual cortex. Mice learn a behavioral task with stimuli that become highly familiar, then are tested on both familiar and novel stimuli. Mice consistently perform the task with novel stimuli, yet responses to stimulus presentations and stimulus omissions are dramatically altered. Further, we find that novelty modifies coding of visual as well as behavioral and task information. At the population level, the direction of these changes is consistent with engagement of the Vip-Sst disinhibitory circuit. At the single cell level, we identify separate clusters of Vip, Sst, and excitatory cells with unique patterns of novelty-induced coding changes. This study and the accompanying open-access dataset reveals the impact of novelty on sensory and behavioral representations in visual cortical circuits and establishes novelty as a key driver of cellular functional diversity.

AbstractThe brain consists of many cell classes yetin vivoelectrophysiology recordings are typically unable to identify and monitor their activity in the behaving animal. Here, we employed a systematic approach to link cellular, multi-modalin vitroproperties from experiments within vivorecorded units via computational modeling and optotagging experiments. We found two one-channel and six multi-channel clusters in mouse visual cortex with distinctin vivoproperties in terms of activity, cortical depth, and behavior. We used biophysical models to map the two one- and the six multi-channel clusters to specificin vitroclasses with unique morphology, excitability and conductance properties that explain their distinct extracellular signatures and functional characteristics. These concepts were tested in ground-truth optotagging experiments with two inhibitory classes unveiling distinctin vivoproperties. This multi-modal approach presents a powerful way to separatein vivoclusters and infer their cellular properties from first principles.

Information is processed by networks of neurons in the brain. On the timescale of sensory processing, those neuronal networks have relatively fixed anatomical connectivity, while functional connectivity, which defines the interactions between neurons, can vary depending on the ongoing activity of the neurons within the network. We thus hypothesized that different types of stimuli, which drive different neuronal activities in the network, could lead those networks to display stimulus-dependent functional connectivity patterns. To test this hypothesis, we analyzed electrophysiological data from the Allen Brain Observatory, which utilized Neuropixels probes to simultaneously record stimulus-evoked activity from hundreds of neurons across 6 different regions of mouse visual cortex. The recordings had single-cell resolution and high temporal fidelity, enabling us to determine fine-scale functional connectivity. Comparing the functional connectivity patterns observed when different stimuli were presented to the mice, we made several nontrivial observations. First, while the frequencies of different connectivity motifs (i.e., the patterns of connectivity between triplets of neurons) were preserved across stimuli, the identities of the neurons within those motifs changed. This means that functional connectivity dynamically changes along with the input stimulus, but does so in a way that preserves the motif frequencies. Secondly, we found that the degree to which functional modules are contained within a single brain region (as opposed to being distributed between regions) increases with increasing stimulus complexity. This suggests a mechanism for how the brain could dynamically alter its computations based on its inputs. Altogether, our work reveals unexpected stimulus-dependence to the way groups of neurons interact to process incoming sensory information.

The visual cortex is hierarchically organized, yet the presence of extensive recurrent and parallel pathways make it challenging to decipher how signals flow between neuronal populations. Here, we tracked the flow of spiking activity recorded from six interconnected levels of the mouse visual hierarchy. By analyzing leading and lagging spike-timing relationships among pairs of simultaneously recorded neurons, we created a cellular-scale directed network graph. Using a module-detection algorithm to cluster neurons based on shared connectivity patterns, we uncovered two multi-regional communication modules distributed across the hierarchy. The direction of signal flow both between and within these modules, differences in layer and area distributions, and distinct temporal dynamics suggest that one module transmits feedforward sensory signals, whereas the other integrates inputs for recurrent processing. These results suggest that multi regional functional modules may be a fundamental feature of organization beyond cortical areas that supports signal propagation across hierarchical recurrent networks.

Humans learn from visual inputs at multiple timescales, both rapidly and flexibly acquiring visual knowledge over short periods, and robustly accumulating online learning progress over longer periods. Modeling these powerful learning capabilities is an important problem for computational visual cognitive science, and models that could replicate them would be of substantial utility in real-world computer vision settings. In this work, we establish benchmarks for both real-time and life-long continual visual learning. Our real-time learning benchmark measures a model's ability to match the rapid visual behavior changes of real humans over the course of minutes and hours, given a stream of visual inputs. Our life-long learning benchmark evaluates the performance of models in a purely online learning curriculum obtained directly from child visual experience over the course of years of development. We evaluate a spectrum of recent deep self-supervised visual learning algorithms on both benchmarks, finding that none of them perfectly match human performance, though some algorithms perform substantially better than others. Interestingly, algorithms embodying recent trends in self-supervised learning - including BYOL, SwAV and MAE - are substantially worse on our benchmarks than an earlier generation of self-supervised algorithms such as SimCLR and MoCo-v2. We present analysis indicating that the failure of these newer algorithms is primarily due to their inability to handle the kind of sparse low-diversity datastreams that naturally arise in the real world, and that actively leveraging memory through negative sampling - a mechanism eschewed by these newer algorithms - appears useful for facilitating learning in such low-diversity environments. We also illustrate a complementarity between the short and long timescales in the two benchmarks, showing how requiring a single learning algorithm to be locally context-sensitive enough to match real-time learning changes while stable enough to avoid catastrophic forgetting over the long term induces a trade-off that human-like algorithms may have to straddle. Taken together, our benchmarks establish a quantitative way to directly compare learning between neural networks models and human learners, show how choices in the mechanism by which such algorithms handle sample comparison and memory strongly impact their ability to match human learning abilities, and expose an open problem space for identifying more flexible and robust visual self-supervision algorithms.

The anatomy of the mammalian visual system, from the retina to the neocortex, is organized hierarchically1. However, direct observation of cellular-level functional interactions across this hierarchy is lacking due to the challenge of simultaneously recording activity across numerous regions. Here we describe a large, open dataset-part of the Allen Brain Observatory2-that surveys spiking from tens of thousands of units in six cortical and two thalamic regions in the brains of mice responding to a battery of visual stimuli. Using cross-correlation analysis, we reveal that the organization of inter-area functional connectivity during visual stimulation mirrors the anatomical hierarchy from the Allen Mouse Brain Connectivity Atlas3. We find that four classical hierarchical measures-response latency, receptive-field size, phase-locking to drifting gratings and response decay timescale-are all correlated with the hierarchy. Moreover, recordings obtained during a visual task reveal that the correlation between neural activity and behavioural choice also increases along the hierarchy. Our study provides a foundation for understanding coding and signal propagation across hierarchically organized cortical and thalamic visual areas.

Temporal continuity of object identity is a feature of natural visual input and is potentially exploited – in an unsupervised manner – by the ventral visual stream to build the neural representation in inferior temporal (IT) cortex. Here, we investigated whether plasticity of individual IT neurons underlies human core object recognition behavioral changes induced with unsupervised visual experience. We built a single-neuron plasticity model combined with a previously established IT population-to-recognition-behavior-linking model to predict human learning effects. We found that our model, after constrained by neurophysiological data, largely predicted the mean direction, magnitude, and time course of human performance changes. We also found a previously unreported dependency of the observed human performance change on the initial task difficulty. This result adds support to the hypothesis that tolerant core object recognition in human and non-human primates is instructed – at least in part – by naturally occurring unsupervised temporal contiguity experience.

Extracellular electrophysiology and two-photon calcium imaging are widely used methods for measuring physiological activity with single-cell resolution across large populations of cortical neurons. While each of these two modalities has distinct advantages and disadvantages, neither provides complete, unbiased information about the underlying neural population. Here, we compare evoked responses in visual cortex recorded in awake mice under highly standardized conditions using either imaging of genetically expressed GCaMP6f or electrophysiology with silicon probes. Across all stimulus conditions tested, we observe a larger fraction of responsive neurons in electrophysiology and higher stimulus selectivity in calcium imaging, which was partially reconciled by applying a spikes-to-calcium forward model to the electrophysiology data. However, the forward model could only reconcile differences in responsiveness when restricted to neurons with low contamination and an event rate above a minimum threshold. This work established how the biases of these two modalities impact functional metrics that are fundamental for characterizing sensory-evoked responses.

The visual cortex is organized hierarchically, but extensive recurrent pathways make it challenging to decipher the flow of information with single neuron resolution. Here, we characterize spiking interactions in populations of neurons from six interconnected areas along the visual hierarchy in awake mice. We generated multi-area, directed graphs of neuronal communication and uncovered two spatially-distributed functional modules. One module is positioned to transmit feedforward sensory signals along the hierarchy, while the other receives convergent input and engages in recurrent processing. The modules differ in layer and area distributions, convergence and divergence, and population-level temporal dynamics. These results reveal a neuronal-resolution cortical network topology in which distinct processing modules are interlaced across multiple areas of the cortical hierarchy.

Cortical circuits can flexibly change with experience and learning, but the effects on specific cell types, including distinct inhibitory types, are not well understood. Here we investigated how excitatory and VIP inhibitory cells in layer 2/3 of mouse visual cortex were impacted by visual experience in the context of a behavioral task. Mice learned a visual change detection task with a set of eight natural scene images. Subsequently, during 2-photon imaging experiments, mice performed the task with these familiar images and three sets of novel images. Strikingly, the temporal dynamics of VIP activity differed markedly between novel and familiar images: VIP cells were stimulus-driven by novel images but were suppressed by familiar stimuli and showed ramping activity when expected stimuli were omitted from a temporally predictable sequence. This prominent change in VIP activity suggests that these cells may adopt different modes of processing under novel versus familiar conditions.