The anterior temporal lobe (ATL) is crucial for learning and storing concepts, yet the inner workings of its multiple cytoarchitectonically distinct subregions remain a 'black box'. Moreover, it is not yet clear how this region interacts with a distributed network of brain regions to access context-specific information. However, two recent papers have made crucial steps forward, while adopting radically different approaches. One (Jackson et al., 2021) used in silico experimentation to ask how a system can meet the core requirements of controlled semantic cognition, comparing multiple theoretical architectures on their ability to meet these demands. The other (Tiesinga et al., 2023) performed in vivo intracortical recordings with unprecedented coverage and spatial precision, comparing cytoarchitectonic ATL subregions. In this perspective, we bridge these recent in silico and in vivo explorations of controlled semantic cognition, demonstrating their remarkable convergence. Thereby, we expose the inner workings of the ATL semantic hub and the regulating effect of frontal regions, while highlighting the crucial next challenges. The result is a neuroanatomically-precise mechanistic model of controlled semantic cognition.

The human visual system consists of a large, yet unknown number of cortical areas. We summarize the efforts which have led to the identification of 19 retinotopic areas in the human occipital cortex, using the macaque visual cortex as a guide. In this process, retinotopic mapping has proven far superior to the study of functional properties. Macaques and humans share early areas (V1, V2, and V3), a motion-sensitive middle temporal (MT/V5) cluster as well as 6 other areas. The remaining human occipital areas either result from the reorganization of a group of monkey areas or seem to be specifically human. Several regions sensitive to motion and even higher-order motion have been described in the parietal cortex, the retinotopic organization of which is still under debate. On the other hand, both dorsal and ventral regions are sensitive to shape, which is most pronounced in the lateral occipital complex (LOC). The anterior part of this complex is flanked by specialized regions devoted to processing faces and bodies. Together these ventral, temporal regions represent the shape of "visual objects" rather than image properties. While the ventral pathway codes the visual identity of objects, recent evidence indicates that the different dorsal stream regions code the visual identity of observed actions classes, as one of the information used by the parietal cortex in the planning of new actions.

The investigation of the perception of others' actions and underlying neural mechanisms has been hampered by the lack of a comprehensive stimulus set covering the human behavioral repertoire. To fill this void, we present a video set showing 100 human actions recorded in natural settings, covering the human repertoire except for emotion-driven (e.g., sexual) actions and those involving implements (e.g., tools). We validated the set using fMRI and showed that observation of the 100 actions activated the well-established action observation network. We also quantified the videos' low-level visual features (luminance, optic flow, and edges). Thus, this comprehensive video set is a valuable resource for perceptual and neuronal studies.

The temporal pole (TP) plays a central role in semantic memory, yet its neural machinery is unknown. Intracerebral recordings in patients discriminating visually the gender or actions of an actor, yielded gender discrimination responses in the ventrolateral (VL) and tip (T) regions of right TP. Granger causality revealed task-specific signals travelling first forward from VL to T, under control of orbitofrontal cortex (OFC) and neighboring prefrontal cortex, and then, strongly, backwards from T to VL. Many other cortical regions provided inputs to or received outputs from both TP regions, often with longer delays, with ventral temporal afferents to VL signaling the actor's physical appearance. The TP response timing reflected more that of the connections to VL, controlled by OFC, than that of the input leads themselves. Thus, visual evidence for gender categories, collected by VL, activates category labels in T, and consequently, category features in VL, indicating a two-stage representation of semantic categories in TP.

An elongated shape, such as an arrow, provides two cues from which its in-plane orientation can be derived: the inclination (first spatial derivative of position) of the overall shape and the position of its endpoints. In the extensive investigation of the perceptual capacities of Davida (Vannuscorps et al., 2022, this journal), those two cues are used interchangeably (experiments 1.1, 1.7, 3.2 and 3.3). Yet at the neuronal level, endpoints require a more complex receptive field (RF) endowed with end-stopping (Heitger et al., 1992; Hubel & Wiesel, 1965; Orban et al., 1979; Schiller et al., 1976; Yazdanbakhsh & Livingstone, 2006), while the inclination of the shape can be analysed by a simple oriented RF (Hubel & Wiesel, 1959). Given the widespread use of gratings to investigate the early visual regions of the non-human primate (nhp) visual system, relatively little is known about processing of endpoints up to area V4 (Pasupathy & Connor, 1999; Ponce et al., 2017). The authors claim that tilt discrimination is spared in Davida (experiments 2.5 and 3.5), but the methods do not state that the position of the lines was randomised to disable the position-of-endpoint cue, allowing the subject to use this alternative cue. It also opens the door to more complex explanations of the deficit, such as conflicts between cues because the position-of-endpoint cue provides erroneous information. In addition, the test involved a comparison between lines, which is more complex than identifying the orientation of a single line (Orban et al., 1984). For example, the temporal comparison in successive orientation discrimination requires the rostral infero-temporal (IT) cortex, where the orientation of the first stimulus is stored in a short-lived buffer (Orban & Vogels, 1998). The use of a single line would have simplified the interpretation, and enabled comparison with nhp lesion studies (De Weerd et al., 1999; Vogels et al., 1997) showing that orientation identification requires only the initial parts of the ventral stream up to V4. The report states (experiment 6.10) that the processing of orientation of kinetic edges has been investigated extensively in human and non-human primates and seems intact. In humans, the so-called kinetic occipital (KO) region (Van Oostende et al., 1997) is in fact part of the lateral occipital (LO) complex (Larsson & Heeger, 2006), which corresponds to dorsal V4 of the monkey (Kolster et al., 2010). Single-cell studies have shown that the extraction of kinetic edges is achieved in IT cortex (Sary et al., 1993), and even in V4 (Mysore et al., 2006), but not in MT (Marcar et al., 1995), nor V1, and only weakly in V2 (Marcar et al., 2000). Even if there is some grouping of neurons according to the cue used in V4, its retinotopic organisation (Gattass et al., 1988) imposes that this segregation is limited to a distance well below the size of a hypothetical focal disruption (e.g. grating patches in V4 are about 1 mm in size, Vanduffel et al., 2002). The kinetic edge data thus suggests that the disruption in processing occurs neither in area V4 nor downstream from it, but rather concerns the white matter (WM) including the connections between areas V2/3 and V4. Not all visual afferents to V4 should be disrupted, as the kinetic edge data suggests that the magnocellular input to V4 is intact. The observation (experiment 2.8) that the processing of tactile orientation is spared, coupled with the single-cell study of Maunsell et al. (1991), reporting a convergence of visual and tactile signals of orientation in 25% of the V4

Objective.Previous studies demonstrated the possibility to fabricate stereo-electroencephalography probes with high channel count and great design freedom, which incorporate macro-electrodes as well as micro-electrodes offering potential benefits for the pre-surgical evaluation of drug resistant epileptic patients. These new polyimide probes allowed to record local field potentials, multi- and single-unit activity (SUA) in the macaque monkey as early as 1 h after implantation, and yielded stable SUA for up to 26 d after implantation. The findings opened new perspectives for investigating mechanisms underlying focal epilepsy and its treatment, but before moving to possible human application, safety data are needed. In the present study we evaluate the tissue response of this new neural interface by assessing post-mortem the reaction of brain tissue along and around the probe implantation site.Approach.Three probes were implanted, independently, in the brain of one monkey (Macaca mulatta) at different times. We used specific immunostaining methods for visualizing neuronal cells and astrocytes, for measuring the extent of damage caused by the probe and for relating it with the implantation time.Main results.The size of the region where neurons cannot be detected did not exceed the size of the probe, indicating that a complete loss of neuronal cells is only present where the probe was physically positioned in the brain. Furthermore, around the probe shank, we observed a slightly reduced number of neurons within a radius of 50µm and a modest increase in the number of astrocytes within 100µm.Significance.In the light of previous electrophysiological findings, the present data suggest the potential usefulness and safety of this probe for human applications.

Action observation is supported by a network of regions in occipito-temporal, parietal, and premotor cortex in primates. Recent research suggests that the parietal node has regions dedicated to different action classes including manipulation, interpersonal interactions, skin displacement, locomotion, and climbing. The goals of the current study consist of: 1) extending this work with new classes of actions that are communicative and specific to humans, 2) investigating how parietal cortex differs from the occipito-temporal and premotor cortex in representing action classes. Human subjects underwent fMRI scanning while observing three action classes: indirect communication, direct communication, and manipulation, plus two types of control stimuli, static controls which were static frames from the video clips, and dynamic controls consisting of temporally-scrambled optic flow information. Using univariate analysis, MVPA, and representational similarity analysis, our study presents several novel findings. First, we provide further evidence for the anatomical segregation in parietal cortex of different action classes: We have found a new site that is specific for representing human-specific indirect communicative actions in cytoarchitectonic parietal area PFt. Second, we found that the discriminability between action classes was higher in parietal cortex than the other two levels suggesting the coding of action identity information at this level. Finally, our results advocate the use of the control stimuli not just for univariate analysis of complex action videos but also when using multivariate techniques.

The posterior parietal cortex (PPC) has long been understood as a high-level integrative station for computing motor commands for the body based on sensory (i.e., mostly tactile and visual) input from the outside world. In the last decade, accumulating evidence has shown that the parietal areas not only extract the pragmatic features of manipulable objects, but also subserve sensorimotor processing of others' actions. A paradigmatic case is that of the anterior intraparietal area (AIP), which encodes the identity of observed manipulative actions that afford potential motor actions the observer could perform in response to them. On these bases, we propose an AIP manipulative action-based template of the general planning functions of the PPC and review existing evidence supporting the extension of this model to other PPC regions and to a wider set of actions: defensive and locomotor actions. In our model, a hallmark of PPC functioning is the processing of information about the physical and social world to encode potential bodily actions appropriate for the current context. We further extend the model to actions performed with man-made objects (e.g., tools) and artifacts, because they become integral parts of the subject's body schema and motor repertoire. Finally, we conclude that existing evidence supports a generally conserved neural circuitry that transforms integrated sensory signals into the variety of bodily actions that primates are capable of preparing and performing to interact with their physical and social world.

Others? observed actions cause continuously changing retinal images, making it challenging to build neural representations of action identity. The monkey anterior intraparietal area (AIP) and its putative human homologue (phAIP) host neurons selective for observed manipulative actions (OMAs). The neuronal activity of both AIP and phAIP allows a stable readout of OMA identity across visual formats, but human neurons exhibit greater invariance and generalize from observed actions to action verbs. These properties stem from the convergence in AIP of superior temporal signals concerning: (i) observed body movements; and (ii) the changes in the body?object relationship. We propose that evolutionarily preserved mechanisms underlie the specification of observed-actions identity and the selection of motor responses afforded by them, thereby promoting social behavior.

Cytoarchitectonic, anatomical and electrophysiological studies have divided the frontal cortex into distinct functional subdivisions. Many of these subdivisions are anatomically connected with the contralateral primary motor cortex (M1); however, effective neurophysiological connectivity between these regions is not well defined in humans. Therefore, we aimed to use dual-coil transcranial magnetic stimulation (TMS) to map, with high spatial resolution, the effective connectivity between different frontal regions of the right hemisphere and contralateral M1 (cM1). TMS was applied over the left M1 alone (test pulse) or after a conditioning pulse was applied to different grid points covering the right frontal cortex, while subjects were at rest, prepared an index finger abduction (Prep-ABD) or precision grip (Prep-PG). MEP motor maps were generated by creating synthetic fMRI volumes, including the normalised MEP values at vertices corresponding to the TMS grid locations. These maps were registered to a common atlas, and statistical parametric mapping was used to identify cortical clusters in which stimulation differentially modulated conditioned MEPs across conditions. We found five significant clusters in the frontal cortex. Three clusters in ventral premotor regions (areas 6v and 44) showed significant differential modulations of contralateral MEPs when rest was compared to Prep-ABD and Prep-PG. Two clusters in rostral dorsolateral prefrontal cortex (areas 8Av and 46) showed differential modulation in MEPs when Prep-ABD was compared to Prep-PG. Our findings demonstrate distinct regions that show task-related connectivity whereby interactions between ventral premotor regions and cM1 differentiate between rest and movement preparation and the dorsolateral prefrontal cortex differentiates primarily between preparation of different types of hand movements. These results thus demonstrate the utility of dual-coil TMS and MEP motor maps to define fine-grained sub-regions in the human frontal cortex, which are functional and causally involved in hand movements. Highlights ### Competing Interest Statement The authors have declared no competing interest.

The processing steps that lead up to a decision, i.e., the transformation of sensory evidence into motor output, are not fully understood. Here, we combine stereoEEG recordings from the human cortex, with single-lead and time-resolved decoding, using a wide range of temporal frequencies, to characterize decision processing during a rule-switching task. Our data reveal the contribution of rostral inferior parietal lobule (IPL) regions, in particular PFt, and the parietal opercular regions in decision processing and demonstrate that the network representing the decision is common to both task rules. We reconstruct the sequence in which regions engage in decision processing on single trials, thereby providing a detailed picture of the network dynamics involved in decision-making. The reconstructed timeline suggests that the supramarginal gyrus in IPL links decision regions in prefrontal cortex with premotor regions, where the motor plan for the response is elaborated.

Humans accurately identify observed actions despite large dy-namic changes in their retinal images and a variety of visual presentation formats. A large network of brain regions in primates participates in the processing of others' actions, with the anterior intraparietal area (AIP) playing a major role in routing information about observed manipulative actions (OMAs) to the other nodes of the network. This study investigated whether the AIP also con-tributes to invariant coding of OMAs across different visual for-mats. We recorded AIP neuronal activity from two macaques while they observed videos portraying seven manipulative actions (drag, drop, grasp, push, roll, rotate, squeeze) in four visual formats. Each format resulted from the combination of two actor's body pos-tures (standing, sitting) and two viewpoints (lateral, frontal). Out of 297 recorded units, 38% were OMA-selective in at least one format. Robust population code for viewpoint and actor's body posture emerged shortly after stimulus presentation, fol-lowed by OMA selectivity. Although we found no fully invariant OMA-selective neuron, we discovered a population code that allowed us to classify action exemplars irrespective of the visual format. This code depends on a multiplicative mixing of signals about OMA identity and visual format, particularly evidenced by a set of units maintaining a relatively stable OMA selectivity across formats despite considerable rescaling of their firing rate depend-ing on the visual specificities of each format. These findings sug-gest that the AIP integrates format-dependent information and the visual features of others' actions, leading to a stable readout of observed manipulative action identity.

When observing others' behavior, it is important to perceive not only the identity of the observed actions (OAs), but also the number of times they were performed. Given the mounting evidence implicating posterior parietal cortex in action observation, and in particular that of manipulative actions, the aim of this study was to identify the parietal region, if any, that contributes to the processing of observed manipulative action (OMA) numerosity, using the functional magnetic resonance imaging technique. Twenty‐one right‐handed healthy volunteers performed two discrimination tasks while in the scanner, responding to video stimuli in which an actor performed manipulative actions on colored target balls that appeared four times consecutively. The subjects discriminated between two small numerosities of either OMAs (“Action” condition) or colors of balls (“Ball” condition). A significant difference between the “Action” and “Ball” conditions was observed in occipito‐temporal cortex and the putative human anterior intraparietal sulcus (phAIP) area as well as the third topographic map of numerosity‐selective neurons at the post‐central sulcus (NPC3) of the left parietal cortex. A further region of interest analysis of the group‐average data showed that at the single voxel level the latter area, more than any other parietal or occipito‐temporal numerosity map, favored numerosity of OAs. These results suggest that phAIP processes the identity of OMAs, while neighboring NPC3 likely processes the numerosity of the identified OAs.

Visual perception of others’ actions is important for social interactions, and the ability to do so, even when one gets only brief glimpses of others’ behavior, may be crucial for survival. At present it is unknown how the human brain solves this problem. Imaging studies have promoted the idea that the multiple demand (MD) system, a domain general system of the human brain, operates in difficult cognitive and perceptual tasks, but not in tasks in which sensory information is reduced. Functional imaging, based on slow hemodynamic responses, may miss or standardize neural events with very diverse time courses. Here we exploited the temporal resolution of stereo-EEG to study directly cortical activity when human subjects must judge the actions of others, but only get brief glimpses of others’ activity, because the videos were truncated. Multiple cortical regions increased their activity in the difficult action discrimination, relative to the easy task when the complete video was visible. The majority of these regions belonged to the MD system, being located in parietal or prefrontal cortex. The variety of time courses, lasting from a few 100ms to several seconds, allowed us to disentangle control from effector regions, the latter processing observed actions. This distinction was further supported by relationships with behavior. A key operation within the control clusters was the prediction of erroneous responses, which was initiated in the PPC soon after the end of the truncated video. The time courses further suggested that MD regions not only exert control, but also perform various evaluations of the effort, important for efficient and intelligent behavior. We observed also increases outside the MD system, in temporo-parietal cortex, which may provide contextual information about variables related to the observed action, such as the actor, the object or the scene. Furthermore, to cope with the brief sensory input, the MD system called upon warning regions reacting to the static picture of the actor presented just before the video. We conclude that discrimination of brief observed actions indeed involves the MD system, which is thus is more general than assumed so far. WE also show that the MD system is more complex than assumed, as it includes evaluation of control, and more flexible, as it interacts with other systems than simply the effector circuit of the task.

The anterior temporal lobe (ATL), located at the tip of the human temporal lobes, has been heavily implicated in semantic processing by neuropsychological and functional imaging studies. These techniques have revealed a hemispheric specialization of ATL, but little about the time scale on which it operates. Here we show that ATL is specifically activated in intracerebral recordings when subjects discriminate the gender of an actor presented in a static frame followed by a video. ATL recording sites respond briefly (100 ms duration) to the visual static presentation of an actor in a task-, but not in a stimulus-duration-dependent way. Their response latencies correlate with subjects’ reaction times, as do their activity levels, but oppositely in the two hemispheres operating in a push-pull fashion. Comparison of ATL time courses with those of more posterior, less specific regions emphasizes the role of inhibitory operations sculpting the fast ATL responses underlying semantic processing.

Introduction Although it has become widely accepted that the action observation network (AON) includes three levels (occipito-temporal, parietal and premotor), little is known concerning the specific role of these levels within perceptual tasks probing action observation. Recent single cell studies suggest that the parietal level carries the information required to discriminate between two-alternative observed actions, but do not exclude possible contributions from the other two levels. Methods Two functional magnetic resonance imaging experiments used a task-based attentional modulation paradigm in which subjects viewed videos of an actor performing a manipulative action on a coloured object, and discriminated between either two observed manipulative actions, two actors or two colours. Results Both experiments demonstrated that relative to actor and colour discrimination, discrimination between observed manipulative actions involved the putative human anterior intraparietal sulcus (phAIP) area in parietal cortex. In one experiment, where the observed actions also differed with regard to effectors, premotor cortex was also specifically recruited. Conclusions Our results highlight the primary role of parietal cortex in discriminating between two-alternative observed manipulative actions, consistent with the view that this level plays a major role in representing the identity of an observed action.

Current knowledge regarding the processing of observed manipulative actions (OMAs) (e.g., grasping, dragging, or dropping) is limited to grasping and underlying neural circuitry remains controversial. Here, we addressed these issues by combining chronic neuronal recordings along the anteroposterior extent of monkeys' anterior intraparietal (AIP) area with tracer injections into the recorded sites. We found robust neural selectivity for 7 distinct OMAs, particularly in the posterior part of AIP (pAIP), where it was associated with motor coding of grip type and own-hand visual feedback. This cluster of functional properties appears to be specifically grounded in stronger direct connections of pAIP with the temporal regions of the ventral visual stream and the prefrontal cortex, as connections with skeletomotor related areas and regions of the dorsal visual stream exhibited opposite or no rostrocaudal gradients. Temporal and prefrontal areas may provide visual and contextual information relevant for manipulative action processing. These results revise existing models of the action observation network, suggesting that pAIP constitutes a parietal hub for routing information about OMA identity to the other nodes of the network.

BACKGROUND/AIMS:To evaluate the perception of three-dimensional (3D) shape in patients with strabismus and the contributions of stereopsis and monocular cues to this perception.METHODS:Twenty-one patients with strabismus with and 20 without stereo acuity as well as 25 age-matched normal volunteers performed two tasks: (1) identifying the closest vertices of 3D shapes from monocular shading (3D-SfS), texture (3D-SfT) or motion cues (3D-SfM) and from binocular disparity (3D-SfD), (2) discriminating 1D elementary features of these cues.RESULTS:Discrimination of the elementary features of luminance, texture and motion did not differ across groups. When the distances between reported and actual closest vertices were resolved into sagittal and frontoparallel plane components, sagittal components in 3D-SfS and frontoparallel components in 3D-SfT indicated larger errors in patients with strabismus without stereo acuity than in normal subjects. These patients could not discriminate one-dimensional elementary features of binocular disparity. Patients with strabismus with stereo acuity performed worse for both components of 3D-SfD and frontoparallel components of 3D-SfT compared with normal subjects. No differences were observed in the perception of 3D-SfM across groups. A comparison between normal subjects and patients with strabismus with normal stereopsis revealed no deficit in 3D shape perception from any cue.CONCLUSIONS:Binocular stereopsis is essential for fine perception of 3D shape, even when 3D shape is defined by monocular static cues. Interaction between these cues may occur in ventral occipitotemporal regions, where 3D-SfS, 3D-SfT and 3D-SfD are processed in the same or neighbouring cortical regions. Our findings demonstrate the perceptual benefit of binocular stereopsis in patients with strabismus.

Coherence is a widely used measure to determine the frequency-resolved functional connectivity between pairs of recording sites, but this measure is confounded by shared inputs to the pair. To remove shared inputs, the 'partial coherence' can be computed by conditioning the spectral matrices of the pair on all other recorded channels, which involves the calculation of a matrix (pseudo-) inverse. It has so far remained a challenge to use the time-resolved partial coherence to analyze intracranial recordings with a large number of recording sites. For instance, calculating the partial coherence using a pseudoinverse method produces a high number of false positives when it is applied to a large number of channels. To address this challenge, we developed a new method that randomly aggregated channels into a smaller number of effective channels on which the calculation of partial coherence was based. We obtained a 'consensus' partial coherence (cPCOH) by repeating this approach for several random aggregations of channels (permutations) and only accepting those activations in time and frequency with a high enough consensus. Using model data we show that the cPCOH method effectively filters out the effect of shared inputs and performs substantially better than the pseudo-inverse. We successfully applied the cPCOH procedure to human stereotactic EEG data and demonstrated three key advantages of this method relative to alternative procedures. First, it reduces the number of false positives relative to the pseudo-inverse method. Second, it allows for titration of the amount of false positives relative to the false negatives by adjusting the consensus threshold, thus allowing the data-analyst to prioritize one over the other to meet specific analysis demands. Third, it substantially reduced the number of identified interactions compared to coherence, providing a sparser network of connections from which clear spatial patterns emerged. These patterns can serve as a starting point of further analyses that provide insight into network dynamics during cognitive processes. These advantages likely generalize to other modalities in which shared inputs introduce confounds, such as electroencephalography (EEG) and magneto-encephalography (MEG).