A multi-modal neural interface capable of long-term recording and stimulation is essential for advancing brain monitoring and developing targeted therapeutics. Among traditional electrophysiological methods, micro-electrocorticography (μECoG) is appealing for chronic applications because it provides a good compromise between invasiveness and high-resolution neural recording. When combining μECoG with optical technologies, such as calcium imaging and optogenetics, this multi-modal approach enables simultaneous recording of neural activity from individual neurons and the ability to perform cell-specific manipulation. While previous efforts have focused on multi-modal interfaces for small animal models, scaling these technologies to larger primate brains remains challenging. In this paper, we present a multi-modal neural interface, named Smart Dura, a functional version of the commonly used artificial dura with integrated recording and stimulation electrodes for large cortical area coverage of the NHP brain. The Smart Dura is fabricated using a thin-film microfabrication process to monolithically integrate a micron-scale electrode array into a soft, flexible, and transparent substrate with high-density electrodes (up to 256 electrodes) while providing matched mechanical compliance with the native tissue and achieving high optical transparency (exceeding 98%). Our in vivo experiments demonstrate electrophysiological recording capabilities combined with neuromodulation, as well as optical transparency that enables structural and functional imaging. This work paves the way toward a chronic neural interface that can provide large-scale, bidirectional interfacing for multi-modal and closed-loop neuromodulation capabilities to study cortical brain activity in non-human primates, with the potential for translation to humans.

Macaque primary visual cortex (V1) exhibits exquisite columnar organization, while midlevel area V4 does not. Here we investigated the functional organization and representational bases of intervening area V2 in three macaques (one male, two females) with high-density Neuropixels recordings and a variety of visual stimuli-shape, texture, drifting grating, and translational motion patches. We observed dense clusters of similarly tuned neurons often spanning ∼500 µm, consistent with a columnar structure. In terms of representational bases, V2 responses were largely explained by stimulus features based on local image statistics: shape tuning is well modeled by a linear combination of orientation filters, and direction selectivity is stronger with surface compared with object motion, in striking contrast to V4. Overall, our results support the progression from columns to sparse clusters as neuronal representations transform from encoding local features and feature conjunctions in V1/V2 to a high-dimensional object-based code in V4.

Multimodal neural interfaces that integrate electrical and optical functionalities are promising tools for neuroscientific and clinical applications that involve recording and manipulating neuronal activity. However, most technologies for multimodal implementation are largely restricted to small animal models and lack the ability to translate to the larger brains of non-human primates (NHPs). Smart Dura, a recently developed large-scale neural interface for NHPs, enables high-density electrophysiological recordings and broad optical accessibility, providing multiscale information with enhanced spatiotemporal resolution. In this paper, the multimodal capabilities of Smart Dura are demonstrated through integration with multiphoton imaging, optical coherence tomography angiography (OCTA), and intrinsic signal optical imaging (ISOI), as well as optical manipulations such as photothrombotic lesioning and optogenetics. Through the Smart Dura, in vivo fluorescence vascular imaging is achieved down to depths of 200 and 550 µm using two-photon and three-photon microscopy, respectively. When combined with simultaneous electrophysiology, Smart Dura also enables the assessment of vascular and neural dynamics via OCTA and ISOI, the induction of ischemic stroke, and the application of optogenetic neuromodulation across a wide cortical area of 20 mm in diameter. These capabilities support comprehensive investigations of brain dynamics in NHPs, advancing translational neurotechnology for human applications.

Multimodal neural interfaces that integrate electrical and optical functionalities are promising tools for neuroscientific and clinical applications that involve recording and manipulating neuronal activity. However, most technologies for multimodal implementation are largely restricted to small animal models and lack the ability to translate to the larger brains of non-human primates (NHPs). Smart Dura, a recently developed large-scale neural interface for NHPs, enables high-density electrophysiological recordings and broad optical accessibility, providing multiscale information with enhanced spatiotemporal resolution. In this paper, the multimodal capabilities of Smart Dura are demonstrated through integration with multiphoton imaging, optical coherence tomography angiography (OCTA), and intrinsic signal optical imaging (ISOI), as well as optical manipulations such as photothrombotic lesioning and optogenetics. Through the transparent Smart Dura, in vivo fluorescence vascular imaging is achieved down to depths of 200 and 550 μm using two-photon and three-photon microscopy, respectively. When combined with simultaneous electrophysiology, Smart Dura also enables assessment of vascular and neural dynamics via OCTA and ISOI, the induction of ischemic stroke, and the application of optogenetic neuromodulation across a wide cortical area of 20 mm in diameter. These capabilities support comprehensive investigations of brain dynamics in NHPs, advancing translational neurotechnology for human applications.

To understand the neural basis of behavior, it is essential to sensitively and accurately measure neural activity at single-neuron and single-spike resolution. Extracellular electrophysiology delivers this, but it has biases in the neurons it detects and it imperfectly resolves their action potentials. To minimize these limitations, we developed a silicon probe with much smaller and denser recording sites than previous designs, called Neuropixels Ultra (NP Ultra). Using NP Ultra, neuronal yield in mouse visual cortex recordings increased by more than 2-fold. With ultra-high spatial resolution, we discovered that a feature of extracellular waveforms, the spatial extent or “footprint,” distinguished axonal from somatic recordings. In addition, three genetically identified cortical cell types could be discriminated from one another with ∼80% accuracy and from other neurons with ∼85% accuracy. NP Ultra improves yield, detection of subcellular compartments, and cell type identification to enable a more powerful dissection of neural circuit activity during behavior.

To understand the neural basis of behavior, it is essential to sensitively and accurately measure neural activity at single neuron and single spike resolution. Extracellular electrophysiology delivers this, but it has biases in the neurons it detects and it imperfectly resolves their action potentials. To minimize these limitations, we developed a silicon probe with much smaller and denser recording sites than previous designs, called Neuropixels Ultra (NP Ultra). This device samples neuronal activity at ultra-high spatial density (~10 times higher than previous probes) with low noise levels, while trading off recording span. NP Ultra is effectively an implantable voltage-sensing camera that captures a planar image of a neuron's electrical field. We use a spike sorting algorithm optimized for these probes to demonstrate that the yield of visually-responsive neurons in recordings from mouse visual cortex improves up to ~3-fold. We show that NP Ultra can record from small neuronal structures including axons and dendrites. Recordings across multiple brain regions and four species revealed a subset of extracellular action potentials with unexpectedly small spatial spread and axon-like features. We share a large-scale dataset of these brain-wide recordings in mice as a resource for studies of neuronal biophysics. Finally, using ground-truth identification of three major inhibitory cortical cell types, we found that these cell types were discriminable with approximately 75% success, a significant improvement over lower-resolution recordings. NP Ultra improves spike sorting performance, detection of subcellular compartments, and cell type classification to enable more powerful dissection of neural circuit activity during behavior.

A paradox exists in our understanding of motion processing in the primate visual system: neurons in the dorsal motion processing stream often strikingly fail to encode long-range and perceptually salient jumps of a moving stimulus. Psychophysical studies suggest that such long-range motion, which requires integration over more distant parts of the visual field, may be based on higher-order motion processing mechanisms that rely on feature or object tracking. Here, we demonstrate that ventral visual area V4, long recognized as critical for processing static scenes, includes neurons that maintain direction selectivity for long-range motion, even when conflicting local motion is present. These V4 neurons exhibit specific selectivity for the motion of objects, i.e., targets with defined boundaries, rather than the motion of surfaces behind apertures, and are selective for direction of motion over a broad range of spatial displacements and defined by a variety of features. Motion direction at a range of speeds can be accurately decoded on single trials from the activity of just a few V4 neurons. Thus, our results identify a novel motion computation in the ventral stream that is strikingly different from, and complementary to, the well-established system in the dorsal stream, and they support the hypothesis that the ventral stream system interacts with the dorsal stream to achieve the higher level of abstraction critical for tracking dynamic objects.

3-photon excitation laser-scanning microscopy enables fluorescence imaging deep in tissue beyond the range of 2-photon microscopy. Here, we report results of functional calcium imaging of neurons with 3-photon microscopy in non-human primates. We compare 2-photon and 3-photon excitation and characterize the performance of 3-photon microscopy at depths beyond the 2-photon limit.

Expansion microscopy and light sheet imaging enable fine-scale resolution of intracellular features that comprise neural circuits. Most current techniques visualize sparsely distributed features across whole brains or densely distributed features within individual brain regions. Here, we visualize dense distributions of immunolabeled proteins across early visual cortical areas in adult macaque monkeys. This process may be combined with multiphoton or magnetic resonance imaging to produce multimodal atlases in large, gyrencephalic brains.

Texture is an important visual attribute for surface pattern discrimination and therefore object segmentation, but the neural bases of texture perception are largely unknown. Previously, we demonstrated that the responses of V4 neurons to naturalistic texture patches are sensitive to four key features of human texture perception: coarseness, directionality, regularity, and contrast. To begin to understand how distinct texture perception emerges from the dynamics of neuronal responses, in 2 macaque monkeys (1 male, 1 female), we investigated the relative contribution of the four texture attributes to V4 responses in terms of the strength and timing of response modulation. We found that the different feature dimensions are associated with different temporal dynamics. Specifically, the response modulation associated with directionality and regularity was significantly delayed relative to that associated with coarseness and contrast, suggesting that the latter are fundamentally simpler feature dimensions. The population of texture-selective neurons could be grouped into multiple clusters based on the combination of feature dimensions encoded, and those subpopulations displayed distinct temporal dynamics characterized by the weighted combinations of multiple features. Finally, we applied a population decoding approach to demonstrate that texture category information can be obtained from short temporal windows across time. These results demonstrate that the representation of different perceptually relevant texture features emerge over time in the responses of V4 neurons. The observed temporal organization provides a framework to interpret how the processing of surface features unfolds in early and midlevel cortical stages, and could ultimately inform the interpretation of perceptual texture dynamics. SIGNIFICANCE STATEMENT To delineate how neuronal responses underlie our ability to perceive visual textures, we related four key perceptual dimensions (coarseness, directionality, regularity, and contrast) of naturalistic textures to the strength and timing of modulation of neuronal responses in area V4, an intermediate stage in the form-processing, ventral visual pathway. Our results provide the first characterization of V4 temporal dynamics for texture encoding along perceptually defined axes.

The Pearson correlation coefficient squared, r2, is an important tool used in the analysis of neural data to quantify the similarity between neural tuning curves. Yet this metric is biased by trial-to-trial variability; as trial-to-trial variability increases, measured correlation decreases. Major lines of research are confounded by this bias, including those involving the study of invariance of neural tuning across conditions and the analysis of the similarity of tuning across neurons. To address this, we extend an estimator, r̂ER2, that was recently developed for estimating model-to-neuron correlation, in which a noisy signal is compared with a noise-free prediction, to the case of neuron-to-neuron correlation, in which two noisy signals are compared with each other. We compare the performance of our novel estimator to a prior method developed by Spearman, commonly used in other fields but widely overlooked in neuroscience, and find that our method has less bias. We then apply our estimator to demonstrate how it avoids drastic confounds introduced by trial-to-trial variability using data collected in two prior studies (macaque, both sexes) that examined two different forms of invariance in the neural encoding of visual inputs—translation invariance and fill-outline invariance. Our results quantify for the first time the gradual falloff with spatial offset of translation-invariant shape selectivity within visual cortical neuronal receptive fields and offer a principled method to compare invariance in noisy biological systems to that in noise-free models. SIGNIFICANCE STATEMENT Quantifying the similarity between two sets of averaged neural responses is fundamental to the analysis of neural data. A ubiquitous metric of similarity, the correlation coefficient, is attenuated by trial-to-trial variability that arises from many irrelevant factors. Spearman recognized this problem and proposed corrected methods that have been extended over a century. We show this method has large asymptotic biases that can be overcome using a novel estimator. Despite the frequent use of the correlation coefficient in neuroscience, consensus on how to address this fundamental statistical issue has not been reached. We provide an accurate estimator of the correlation coefficient and apply it to gain insight into visual invariance.

Signal correlation (rs) is commonly defined as the correlation between the tuning curves of two neurons and is widely used as a metric of tuning similarity. It is fundamental to how populations of neurons represent stimuli and has been central to many studies of neural coding. Yet the classic estimate, Pearson's correlation coefficient, r̂s, between the average responses of two neurons to a set of stimuli suffers from confounding biases. The estimate r̂s can be downwardly biased by trial-to-trial variability and also upwardly biased by trial-to-trial correlation between neurons, and these biases can hide important aspects of neural coding. Here we provide analytic results on the source of these biases and explore them for ranges of parameters that are relevant for electrophysiological experiments. We then provide corrections for these biases that we validate in simulation. Furthermore, we apply these corrected estimators to make the following novel experimental observation in cortical area MT: pairs of nearby neurons that are strongly tuned for motion direction tend to have high signal correlation, and pairs that are weakly tuned tend to have low signal correlation. We dismiss a trivial explanation for this and find that an analogous trend holds for orientation tuning in the primary visual cortex. We also consider the potential consequences for encoding whereby the association of signal correlation and tuning strength naturally regularizes the dimensionality of downstream computations. SIGNIFICANCE STATEMENT Fundamental to how cortical neurons encode information about the environment is their functional similarity, that is, the redundancy in what they encode and their shared noise. These properties have been extensively studied theoretically and experimentally throughout the nervous system, but here we show that a common estimator of functional similarity has confounding biases. We characterize these biases and provide estimators that do not suffer from them. Using our improved estimators, we demonstrate a novel result, that is, there is a positive relationship between tuning curve similarity and amplitude for nearby neurons in the visual cortical motion area MT. We provide a simple stochastic model explaining this relationship and discuss how it would naturally regularize the dimensionality of neural encoding.

The Pearson correlation coefficient squared, r 2, is often used in the analysis of neural data to estimate the relationship between neural tuning curves. Yet this metric is biased by trial-to-trial variability: as trial-to-trial variability increases, measured correlation decreases. Major lines of research are confounded by this bias, including the study of invariance of neural tuning across conditions and the similarity of tuning across neurons. To address this, we extend the estimator, ![Graphic][1]</img>, developed for estimating model-to-neuron correlation to the neuron-to-neuron case. We compare the estimator to a prior method developed by Spearman, commonly used in other fields but widely overlooked in neuroscience, and find that our method has less bias. We then apply our estimator to the study of two forms of invariance and demonstrate how it avoids drastic confounds introduced by trial-to-trial variability. Significant Statement Quantifying the similarity between two sets of averaged neural responses is fundamental to the analysis of neural data. A ubiquitous metric of similarity, the correlation coefficient, is attenuated by trial-to-trial variability that depends on a variety of irrelevant factors. Spearman recognized this problem and proposed corrected methods that have been extended over a century. We show this method has large asymptotic biases and derive a novel estimator to overcome this. Despite the frequent use of the correlation coefficient in neuroscience, consensus on how to address this fundamental statistical issue has not been reached. We both explicate this issue in a neuroscience setting while at the same time making major strides in addressing it. ### Competing Interest Statement The authors have declared no competing interest. [1]: /embed/inline-graphic-1.gif

The correlation coefficient squared, r2, is commonly used to validate quantitative models on neural data, yet it is biased by trial-to-trial variability: as trial-to-trial variability increases, measured correlation to a model's predictions decreases. As a result, models that perfectly explain neural tuning can appear to perform poorly. Many solutions to this problem have been proposed, but no consensus has been reached on which is the least biased estimator. Some currently used methods substantially overestimate model fit, and the utility of even the best performing methods is limited by the lack of confidence intervals and asymptotic analysis. We provide a new estimator, [Formula: see text], that outperforms all prior estimators in our testing, and we provide confidence intervals and asymptotic guarantees. We apply our estimator to a variety of neural data to validate its utility. We find that neural noise is often so great that confidence intervals of the estimator cover the entire possible range of values ([0, 1]), preventing meaningful evaluation of the quality of a model's predictions. This leads us to propose the use of the signal-to-noise ratio (SNR) as a quality metric for making quantitative comparisons across neural recordings. Analyzing a variety of neural data sets, we find that up to ∼ 40% of some state-of-the-art neural recordings do not pass even a liberal SNR criterion. Moving toward more reliable estimates of correlation, and quantitatively comparing quality across recording modalities and data sets, will be critical to accelerating progress in modeling biological phenomena.

We compute the firing rate of a leaky integrate-and-fire (LIF) neuron with stochastic conductance-based inputs in the limit when synaptic decay times are much shorter than the membrane time constant. A comparison of our analytical results to numeric simulations is presented for a range of biophysically-realistic parameters.

Blurry visual scenes arise from many causes and image blur is known to be important for scene interpretation, yet there have been very few studies of the visual encoding of blur in biological visual systems or in artificial visual systems trained for object recognition. Recently, a study of single neurons in the visual cortex of macaque monkeys found that a significant fraction of neurons were more responsive to blurred visual stimuli than to sharply defined stimuli. This raises two questions: (1) what types of visual features in natural scenes might underlie blur selectivity in macaque cortex, and (2) can blur selectivity be found in artificial neural networks using the simple artificial stimuli previously applied in vivo? To answer these questions, we presented simple shape stimuli to the widely studied deep convolutional neural network (CNN) known as AlexNet and used deconv-net visualization to identify features that are critical for driving blur selective units. We found that a substantial number of units in the CNN were selective for blur and that several categories of blur selectivity emerged in early-to-middle processing stages. Prominent among these is a set of units selective for spatial boundaries defined by blur contrast. These blur-contrast boundary units may serve the important task of segmentation in natural photographic images and may relate to a large body of literature on second-order boundary detection. Our results lead to the prediction that such units could exist in the visual cortex but have yet to be well characterized and localized, and they provide direction for future neurophysiological tests of blur selectivity in vivo.

Visual area V4 is an important midlevel cortical processing stage that subserves object recognition in primates. Studies investigating shape coding in V4 have largely probed neuronal responses with filled shapes, i.e., shapes defined by both a boundary and an interior fill. As a result, we do not know whether form-selective V4 responses are dictated by boundary features alone or if interior fill is also important. We studied 43 V4 neurons in two male macaque monkeys ( Macaca mulatta) with a set of 362 filled shapes and their corresponding outlines to determine how interior fill modulates neuronal responses in shape-selective neurons. Only a minority of neurons exhibited similar response strength and shape preferences for filled and outline stimuli. A majority responded preferentially to one stimulus category (either filled or outline shapes) and poorly to the other. Our findings are inconsistent with predictions of the hierarchical-max (HMax) V4 model that builds form selectivity from oriented boundary features and takes little account of attributes related to object surface, such as the phase of the boundary edge. We modified the V4 HMax model to include sensitivity to interior fill by either removing phase-pooling or introducing unoriented units at the V1 level; both modifications better explained our data without increasing the number of free parameters. Overall, our results suggest that boundary orientation and interior surface information are both maintained until at least the midlevel visual representation, consistent with the idea that object fill is important for recognition and perception in natural vision. NEW & NOTEWORTHY The shape of an object's boundary is critical for identification; consistent with this idea, models of object recognition predict that filled and outline versions of a shape are encoded similarly. We report that many neurons in a midlevel visual cortical area respond differently to filled and outline shapes and modify a biologically plausible model to account for our data. Our results suggest that representations of boundary shape and surface fill are interrelated in visual cortex.

A central goal in the study of the primate visual cortex and hierarchical models for object recognition is understanding how and why single units trade off invariance versus sensitivity to image transformations. For example, in both deep networks and visual cortex there is substantial variation from layer-to-layer and unitto-unit in the degree of translation invariance. Here, we provide theoretical insight into this variation and its consequences for encoding in a deep network. Our critical insight comes from the fact that rectification simultaneously decreases response variance and correlation across responses to transformed stimuli, naturally inducing a positive relationship between invariance and dynamic range. Invariant input units then tend to drive the network more than those sensitive to small image transformations. We discuss consequences of this relationship for AI: deep nets naturally weight invariant units over sensitive units, and this can be strengthened with training, perhaps contributing to generalization performance. Our results predict a signature relationship between invariance and dynamic range that can now be tested in future neurophysiological studies.

The distinct visual sensations of shape and texture have been studied separately in cortex; therefore, it remains unknown whether separate neuronal populations encode each of these properties or one population carries a joint encoding. We directly compared shape and texture selectivity of individual V4 neurons in awake macaques (1 male, 1 female) and found that V4 neurons lie along a continuum from strong tuning for boundary curvature of shapes to strong tuning for perceptual dimensions of texture. Among neurons tuned to both attributes, tuning for shape and texture were largely separable, with the latter delayed by ∼30 ms. We also found that shape stimuli typically evoked stronger, more selective responses than did texture patches, regardless of whether the latter were contained within or extended beyond the receptive field. These results suggest that there are separate specializations in mid-level cortical processing for visual attributes of shape and texture. SIGNIFICANCE STATEMENT Object recognition depends on our ability to see both the shape of the boundaries of objects and properties of their surfaces. However, neuroscientists have never before examined how shape and texture are linked together in mid-level visual cortex. In this study, we used systematically designed sets of simple shapes and texture patches to probe the responses of individual neurons in the primate visual cortex. Our results provide the first evidence that some cortical neurons specialize in processing shape whereas others specialize in processing textures. Most neurons lie between the ends of this continuum, and in these neurons we find that shape and texture encoding are largely independent.