The congruency sequence effect (CSE) refers to a modulation of current-trial interference by the preceding trial in conflict tasks, such as the Eriksen flanker task, with interference typically reduced after incongruent relative to congruent trials. Although widely attributed to dynamic adjustments in cognitive control, the neural mechanisms and temporal dynamics underlying this effect remain poorly understood. Here, we combined a release-and-press flanker task with EEG decoding to examine how trial congruency and response type shape behavior and neural processing in healthy adults. Behaviorally, the CSE emerged only when responses repeated, highlighting the dominant role of stimulus-response binding over abstract control mechanisms. Neural decoding mirrored the CSE: current-trial congruency was more reliably decoded on repeated-response trials following congruent versus incongruent trials. This neural CSE was stimulus-locked, occurred between 450 and 550 ms post-stimulus onset, and was driven by theta-band activity. More broadly, congruency decoding was particularly robust over frontal channels while cross-temporal generalization indicated transient, sequential neural representations underlying control signals. Together, these findings demonstrate that both behavioral and neural signatures of the CSE are tightly constrained by response repetition and emerge within a narrow temporal window. Reach-based measures and multivariate EEG decoding jointly provide a fine-grained account of when, where, and under what conditions control-related signals unfold after conflict.

Expression recognition relies on the ability to distinguish subtle visual differences across a range of facial expressions. Here, we examine the neural representation of dynamic expressions as reflected by electroencephalography (EEG) data in human adults. We find that a wide range of expressions (i.e., 14 emotional and 10 conversational expressions) can be decoded from neural signals, and that their representational structure evinces the classic dimensions of valence and arousal. Critically, we recover, through EEG-based video reconstruction, dynamic representations whose content succeeds in capturing even fine differences across related expressions (e.g., happy-satiated versus schadenfreude). Further, time-resolved decoding reveals anticipatory dynamics that maximize accuracy before the occurrence of an apex expression in the visual stimulus. These results are validated against behavioral data, which yield static reconstructions consistent with their neural counterparts. Thus, our results shed light on the representational basis of expression recognition and serve to recover the dynamic content of visual experience.

Self-representations are central to cognition, yet their internal structure and visual fidelity remain poorly understood. We combined behavioral and computational approaches to characterize self-face representations from perception and memory by linking similarity judgments, model alignment, and image reconstruction. Participants provided similarity judgments for self, familiar, and unfamiliar faces, which were compared against the representational structure of artificial neural networks (ANNs). We then used behavior-based image reconstruction to visualize the image-level consequences of those similarity structures. Self-face judgments aligned reliably with recognition-trained, identity-separating ANNs, but not with the tested generative latent spaces. Pixelwise similarity also explained unique variance, indicating that these judgments contained both identity-level and pictorial information. Consistent with this pattern, reconstructions recovered self-face information from both perception and memory. These reconstructions preserved identity-relevant structure but were less constrained by the target image than reconstructions of other faces, with image-specific fidelity weakest for recalled self-faces. Exploratory bias analyses suggested that self-beliefs modulate these representations, especially in memory: higher self-rated attractiveness was associated with more attractive reconstructed self-faces, and higher self-concept clarity was modestly associated with more accurate representations. Together, these findings indicate that self-face representations constitute visually recoverable yet systematically biased representations, shaped by both objective appearance and self-belief-related information.

The other-race effect (ORE) is the disadvantage of recognizing faces of another race than one’s own. While its prevalence is behaviorally well documented, the representational basis of ORE remains unclear. This study employs StyleGAN2, a deep learning technique for generating photorealistic images to uncover face representations and to investigate ORE’s representational basis. To this end, we collected pairwise visual similarity ratings with same- and other-race faces across East Asian and White participants exhibiting robust levels of ORE. Leveraging the significant overlap in representational similarity between the GAN’s latent space and perceptual representations in human participants, we designed an image reconstruction approach aiming to reveal internal face representations from behavioral similarity data. This methodology yielded hyper-realistic depictions of face percepts, with reconstruction accuracy well above chance, as well as an accuracy advantage for same-race over other-race reconstructions, which mirrored ORE in both populations. Further, a comparison of reconstructions across participant race revealed a novel age bias, with other-race face reconstructions appearing younger than their same-race counterpart. Thus, our work proposes a new approach to exploiting the utility of GANs in image reconstruction and provides new avenues in the study of ORE.

The other-race effect (ORE) refers to poorer recognition for faces of other races than one’s own. This study investigates the neural and representational basis of ORE in East Asian and White participants using behavioral measures, neural decoding, and image reconstruction based on electroencephalography (EEG) data. Our investigation identifies a reliable neural counterpart of ORE, with reduced decoding accuracy for other-race faces, and it relates this result to higher density of other-race face representations in face space. Then, we characterize the temporal dynamics and the prominence of ORE for individual variability at the neural level. Importantly, we use a data-driven image reconstruction approach to reveal visual biases underlying other-race face perception, including a tendency to perceive other-race faces as more typical, younger, and more expressive. These findings provide neural evidence for a classical account of ORE invoking face space compression for other-race faces. Further, they indicate that ORE involves not only reduced identity information but also broader, systematic distortions in visual representation with considerable cognitive and social implications.

Recent research shows that the intention to act on an object alters its neural representation in ways as afforded by underlying sensorimotor processes. For example, the intention to grasp and pick up an object results in representations of the object's weight. But these representations become grasp-specific only immediately before object lift if weight information is relayed through object material. This feature triggers earlier representations regardless of intention probably because material-weight contingencies are overlearned. In contrast, recently learned weight cues should be recalled deliberately during grasp planning resulting in early grasp-specific representations. Here, we examined how action intentions affect the representation of newly acquired color-weight contingencies. We recorded electroencephalography while human participants grasped or reached for objects that varied in shape and density as indicated by their color. Multivariate analyses revealed a grasp-specific reactivation of color during planning that was mirrored in beta band. This suggests that task relevancy influences the representation of color such that previously encoded color-weight contingencies may be reactivated as required for grasping, mediated top-down via working memory. Grasp-specific representations of shape and color were also present in theta band, perhaps reflecting attentional activity. These results provide novel insights into the interplay between cognition and motor planning processes.

Self-representations are central to cognition, yet their internal structure and visual fidelity remain poorly understood. Here, we combine behavioral and computational approaches to recover and assess visual self-face representations from both perception and memory. Participants provided similarity judgments for self, familiar, and unfamiliar faces, which were compared against the representational structure of artificial neural networks (ANNs). We further applied behavior-based image reconstruction to visualize internal representations. Discriminative ANNs aligned more closely with self-face similarity judgments than generative models, suggesting that self-representations emphasize high-level, identity-invariant features while sacrificing lower-level visual detail. Image reconstruction confirmed that self-face representations can be recovered from both perception and memory, though with reduced fidelity relative to familiar and unfamiliar faces. Exploratory analyses further indicated how self-beliefs may modulate these representations: higher self-concept clarity was associated with more accurate representations and self-attractiveness ratings with more attractive ones. Together, these findings indicate that self-face representations constitute a coherent but systematically distorted internal image, shaped by both objective appearance and self-beliefs. By extending image reconstruction to the domain of self-representations, this work provides a novel methodological framework for quantifying their content and underlying distortions.

Face ensemble encoding involves synthesizing summary information from groups of faces, providing a mechanism to overcome limitations in visual working memory. Yet, research on the role of attention has revealed mixed findings. Also, the simultaneous processing of summary representations and individual faces within an ensemble remains largely unexplored. Here, across three experiments, participants viewed ensembles without a central face (n = 32), or with a central face, while attention was distributed across the entire ensemble (n = 38) or focused centrally (n = 38). Critically, the consistency of center and surround faces varied as a function of emotional valence (i.e., same versus opposite). Participants completed an expression similarity-rating task between an ensemble and a single face, which was used to recover, via image reconstruction, visual estimates of summary representations. Reconstructions were then assessed against central faces, surrounding faces, and their averages. We show that focused attention enhances central face representation and that consistency benefits the representation of both the center and of the surround. However, central faces outweigh the overall surround representation only when attention is focused on a center face inconsistent with its surround. These findings reveal a flexible relationship between attention and stimulus structure in ensemble perception.

We present Alljoined1, a dataset built specifically for EEG-to-Image decoding. Recognizing that an extensive and unbiased sampling of neural responses to visual stimuli is crucial for image reconstruction efforts, we collected data from 8 participants looking at 10,000 natural images each. We have currently gathered 46,080 epochs of brain responses recorded with a 64-channel EEG headset. The dataset combines response-based stimulus timing, repetition between blocks and sessions, and diverse image classes with the goal of improving signal quality. For transparency, we also provide data quality scores. We publicly release the dataset and all code at https://linktr.ee/alljoined1.

The intention to act influences the computations of various task-relevant features. However, little is known about the time course of these computations. Furthermore, it is commonly held that these computations are governed by conjunctive neural representations of the features. But, support for this view comes from paradigms arbitrarily combining task features and affordances, thus requiring representations in working memory. Therefore, the present study used electroencephalography and a well-rehearsed task with features that afford minimal working memory representations to investigate the temporal evolution of feature representations and their potential integration in the brain. Female and male human participants grasped objects or touched them with a knuckle. Objects had different shapes and were made of heavy or light materials with shape and weight being relevant for grasping, not for “knuckling.” Using multivariate analysis showed that representations of object shape were similar for grasping and knuckling. However, only for grasping did early shape representations reactivate at later phases of grasp planning, suggesting that sensorimotor control signals feed back to the early visual cortex. Grasp-specific representations of material/weight only arose during grasp execution after object contact during the load phase. A trend for integrated representations of shape and material also became grasp-specific but only briefly during the movement onset. These results suggest that the brain generates action-specific representations of relevant features as required for the different subcomponents of its action computations. Our results argue against the view that goal-directed actions inevitably join all features of a task into a sustained and unified neural representation.

Extensive work has investigated the neural processing of single faces, including the role of shape and surface properties. However, much less is known about the neural basis of face ensemble perception (e.g., simultaneously viewing several faces in a crowd). Importantly, the contribution of shape and surface properties have not been elucidated in face ensemble processing. Furthermore, how single central faces are processed within the context of an ensemble remains unclear. Here, we probe the neural dynamics of ensemble representation using pattern analyses as applied to electrophysiology data in healthy adults (seven males, nine females). Our investigation relies on a unique set of stimuli, depicting different facial identities, which vary parametrically and independently along their shape and surface properties. These stimuli were organized into ensemble displays consisting of six surround faces arranged in a circle around one central face. Overall, our results indicate that both shape and surface properties play a significant role in face ensemble encoding, with the latter demonstrating a more pronounced contribution. Importantly, we find that the neural processing of the center face precedes that of the surround faces in an ensemble. Further, the temporal profile of center face decoding is similar to that of single faces, while those of single faces and face ensembles diverge extensively from each other. Thus, our work capitalizes on a new center-surround paradigm to elucidate the neural dynamics of ensemble processing and the information that underpins it. Critically, our results serve to bridge the study of single and ensemble face perception.

The other-race effect, a disadvantage at recognizing faces of other races than one’s own, has received considerable attention, especially regarding its wide scope and underlying mechanisms. Here, we aim to elucidate its neural and representational basis by relating behavioral performance in East Asian and White individuals to neural decoding and image reconstruction relying on electroencephalography data. Our investigation uncovers a reliable neural counterpart of the other-race effect (i.e., a decoding disadvantage for other-race faces) along with its extended dynamics and prominence across individuals. Further, it retrieves, via neural-based image reconstruction, visual representations underlying other-race face perception and their intrinsic biases. Notably, our data-driven approach reveals that other-race faces are perceived not just as more typical but, also, as younger and more expressive. These findings, pointing to multiple visual biases surrounding the other-race effect, speak to the complexity of its neural mechanisms and its social implications. ### Competing Interest Statement The authors have declared no competing interest.

Recent research shows that the intention to act on an object alters its neural representation in ways as afforded by underlying sensorimotor processes. For example, the intention to grasp and pick up an object results in representations of the object`s weight. But these representations become grasp-specific only immediately before object lift if weight information is relayed through object material. This feature triggers earlier representations regardless of intention probably because material-weight contingencies are overlearned. By contrast, recently learned weight cues should be recalled deliberately during grasp planning resulting in early grasp-specific representations. Here, we examined how action intentions affect the representation of newly acquired colour-weight contingencies. We recorded electroencephalography while human participants grasped or reached for objects that varied in shape and density as indicated by their colour. Multivariate analyses revealed a grasp-specific reactivation of colour during planning that was mirrored in beta band. This suggests that task-relevancy influences the representation of colour such that previously encoded colour-weight contingencies may be reactivated as required for grasping, mediated top-down via working memory. Grasp-specific representations of shape and colour were also present in theta band, perhaps reflecting attentional activity. These results provide novel insights into the interplay between cognition and motor planning processes. ### Competing Interest Statement The authors have declared no competing interest.

Personality traits and affective states are associated with biases in facial emotion perception. However, the precise personality impairments and affective states that underlie these biases remain largely unknown. To investigate how relevant factors influence facial emotion perception and recollection, Experiment 1 employed an image reconstruction approach in which community-dwelling adults (N = 89) rated the similarity of pairs of facial expressions, including those recalled from memory. Subsequently, perception- and memory-based expression representations derived from such ratings were assessed across participants and related to measures of personality impairment, state affect, and visual recognition abilities. Impairment in self-direction and level of positive affect accounted for the largest components of individual variability in perception and memory representations, respectively. Additionally, individual differences in these representations were impacted by face recognition ability. In Experiment 2, adult participants (N = 81) rated facial image reconstructions derived in Experiment 1, revealing that individual variability was associated with specific visual face properties, such as expressiveness, representation accuracy, and positivity/negativity. These findings highlight and clarify the influence of personality, affective state, and recognition abilities on individual differences in the perception and recollection of facial expressions.

Damage to the medial temporal lobe (MTL), which is traditionally considered to subserve memory exclusively, has been reported to contribute to impaired face perception. However, it remains unknown how exactly such brain lesions may impact face representations and in particular facial shape and surface information, both of which are crucial for face perception. The present study employed a behavioral-based image reconstruction approach to reveal the pictorial representations of face perception in two amnesic patients: DA, who has an extensive bilateral MTL lesion that extends beyond the MTL in the right hemisphere, and BL, who has damage to the hippocampal dentate gyrus (DG). Both patients and their respective matched controls completed similarity judgments for pairs of faces, from which facial shape and surface features were subsequently derived and synthesized to create images of reconstructed facial appearance. Participants also completed a face oddity judgment task (FOJT) that has previously been shown to be sensitive to MTL cortical damage. While BL exhibited an impaired pattern of performance on the FOJT, DA demonstrated intact performance accuracy. Notably, the recovered pictorial content of faces was comparable between both patients and controls, although there was evidence for atypical face representations in BL particularly with regards to color. Our work provides novel insight into the face representations underlying face perception in two well-studied amnesic patients in the literature and demonstrates the applicability of the image reconstruction approach to individuals with brain damage.

Clarifying the neural and representational basis of mental imagery has elicited significant interest in the study of visual recognition. Recently, numerous attempts have been directed at uncovering the structure and the content of visual imagery. However, these attempts have mostly targeted simple visual features (e.g., orientations, shapes, or single letters), limiting the theoretical and practical implications of this research. To address these limitations, the current study aimed to decode and to reconstruct the appearance of single words from mental imagery with the aid of functional magnetic resonance imaging (fMRI). We collected fMRI data from 13 healthy right-handed adults while they passively viewed or mentally imagined the appearance of three-letter concrete nouns with a consonant-vowel-consonant structure. Consistent with previous findings, multivariate analyses demonstrated that pairs of words can be discriminated from neural patterns when words are viewed and, also, when they are imagined. However, decoding relied more extensively on early visual areas in the former case, for perception, and more extensively on higher-level visual areas, such as the visual word form area (vWFA), in the latter case, for imagery. To assess and to visualize the representational content underlying successful decoding, imagery-based image reconstruction was conducted by mapping the neural patterns of visual words during imagery onto a representational feature space extracted from neural signals during perception. This analysis revealed successful levels of imagery-based image reconstruction for single words in the early visual cortex as well as in the vWFA. Thus, our findings speak to overlapping neural representations between imagery and perception, both in low-level visual areas and higher-order visual cortex. Further, they shed light on the fine-grained neural representations of visual-orthographic information during mental imagery.