Emotion is typically perceived in dynamic multisensory contexts. Although previous studies have demonstrated multimodal gains in audiovisual recognition (i.e., general enhancement in audiovisual conditions), they have not clarified how visual and auditory information contribute to affective judgments or whether their contribution differs by emotion categories. The current study investigated how emotion perception varies across modalities and examined whether these variations depend on emotion categories. We conducted an emotion perception experiment using audio-only (AO), video-only (VO), and audio-visual (AV) speech stimuli. Participants evaluated the valence (positive/negative) and arousal (weak/strong) of stimuli depicting seven emotions (angry, calm, disgust, fearful, happy, sad, and surprise). Overall, AO stimuli tended to be perceived as more negative in valence and weaker in arousal compared to VO or AV stimuli. In representational similarity analysis, the response patterns for AV were more similar to VO than to AO. Happy and disgust stimuli were rated as weaker in arousal and less positive or less negative in valence in AO condition than in other conditions. For surprise, VO stimuli were rated more negatively than AO and AV. However, angry, fearful, and sad emotions did not differ between modalities. These findings indicate that affective judgments are not solely driven by general enhancement from combined audiovisual cues; rather, the relative importance of sensory modalities varies by emotion, suggesting that specific unimodal cues can be as informative as multimodal information. These modality-specific contributions to emotion perception have implications for clinical assessment, social communication, and affective neuroscience.
This study investigated how stimulus material and effector specificity influence interference control (a form of inhibitory control) in adolescent players. Ninety-one males aged 15–18 years (28 handball players, 34 soccer players, 29 non-player controls) completed nine modified Flanker task versions varying in stimulus type (abstract, handball-specific, soccer-specific) and response modality (finger, hand, foot). Mixed-design ANOVAs revealed significant main effects of congruency, stimulus type, and response modality. Responses were slower, and interference effects were larger in incongruent than in congruent trials. In addition, sports-specific stimuli and more motorically demanding response modalities were associated with increased response times compared to abstract arrow stimuli and simple finger responses. A congruency × stimulus interaction revealed reduced flanker effects for sports-specific stimuli, indicating that domain-specificity shapes interference processing. However, no consistent group differences were found, suggesting adolescent players do not exhibit broad cognitive advantages, or such advantages are context-dependent. Overall, the findings suggest that performance was primarily shaped by perceptual and motor requirements of the task, i.e., bystimulus complexity and response modality rather than by differences between athlete and control groups. Longitudinal research is needed to clarify how training and development interact to shape cognitive–motor control.
Co-attention has been shown to enhance memory. Although this effect has been replicated in different social contexts, its minimal conditions remain unclear. Features of previous experiments have been conducive to collaboration, such as participants completing complementary tasks or experiencing a minimal ingroup manipulation. Consequently, it remains unknown whether co-attention effects on memory reflect a fundamental motivation to represent the mental states of those around us, or whether they are limited to coordination contexts. The proposed studies will test whether mere passive co-attention with a stranger is sufficient to enhance memory. We plan two experiments to: (1) conceptually replicate Shteynberg’s (2010; JPSP) finding that seeing the same (versus different) stimuli alongside an in-group member enhances memory, and (2) test whether mere co-attention enhances memory, absent the group manipulation.
Smooth pursuit eye movements can offer a window into the interaction between sensorimotor control and cognition. The present study examined how pursuit might be affected by dual-task demands varying in sensory modality and cross-modal conflict. A second, exploratory aim was to assess whether individual differences in working memory (WM) capacity could moderate these effects. Participants were asked to track a moving target whilst performing mental calculations on visual, auditory, or bimodal digit streams. Pursuit was stabilised when attention was directed to visual input but substantially destabilised when directed to auditory input, revealing a bidirectional modality asymmetry. When attention was focused on an auditory stream in the presence of incongruent visual digits, cognitive error increased whilst pursuit variability did not, suggesting that additional conflict may tax cognitive rather than oculomotor processes. Higher WM capacity appeared to provide a modest, selective buffer, reducing pursuit variability specifically under demanding auditory conditions. We interpret these findings through a Selection-with-Support (SWS) principle: dual-task costs depend upon the representational alignment between attentional selection and the motor circuits supporting pursuit. Visual attention appears to directly support pursuit by engaging shared sensorimotor circuits, whilst auditory attention, lacking such premotor grounding, may rely upon executive control that WM can only partially sustain. The present study therefore offers a plausible mechanistic account of how attention might be motorically grounded.
Individuals routinely learn patterns from sights and sounds, yet it remains unclear how memories of those patterns compare across senses over time. Statistical learning, the implicit detection and extraction of patterns from continuous input, has been documented across sensory modalities, but most work has focused on immediate learning in unisensory contexts. This study, using a within-subject design, directly compared 24-hour consolidation trajectories of auditory, visual, and audiovisual statistical learning. Twenty-six young adults completed three statistical learning tasks across auditory, visual, and audiovisual modalities. In each task, participants were exposed to a continuous stream of stimuli with embedded triplet patterns. Learning of these patterns was tested immediately after exposure and again 24 h later. Across immediate and delayed tests, multisensory exposure yielded higher accuracy than unisensory exposure, demonstrating a robust multisensory-over-unisensory advantage. In contrast, only auditory statistical learning showed overnight gains, whereas visual and multisensory learning remained stable, suggesting a consolidation advantage for auditory input. Cross-task correlations, both at immediate test and for consolidation magnitudes, were weak, suggesting substantial modality-specific contributions to statistical learning. These findings provide the first evidence that multisensory input boosts long-term retention of learned patterns, and the consolidation of these patterns operates in a modality-specific manner. Individuals readily learn and remember patterns from sights and sounds, but how well these memories persist across senses is unclear. We compared auditory, visual, and audiovisual pattern learning within young adults. Audiovisual presentation increased test performance relative to either sense alone. However, only auditory pattern learning improved overnight. Visual and audiovisual learning remained stable. Correlations were small within sessions and for overnight change, highlighting that consolidation of newly learned patterns largely depends on specific sensory modalities rather than a single, shared process across modalities.
