Under most circumstances, we can rely visual information to quickly and accurately discriminate “real” objects (e.g., fresh fruit) from “fake” objects (e.g., plastic fruit). It is unclear, however, whether this distinction is made early along the ventral visual stream when basic object features such as colour (e.g., primary visual cortex; V1) and texture (e.g., collateral sulcus; COS) are being processed, or whether information regarding object authenticity is extracted in later visual or memory regions (e.g., perirhinal cortex, lateral occipital cortex). To examine this question, participants were placed in an fMRI scanner, and presented with 300 objects photographed in colour or greyscale. Half of the objects were fake, and the other half were real. The participant’s task was to categorise each image as presenting either a real or fake object. Broadly, our analyses revealed significant activation in CoS when participants categorised real objects, particularly when they were presented in colour. We also observed activation in V1 for coloured objects, particularly real ones. These results suggest that our seemingly intuitive ability to rapidly discriminate real from fake objects occurs at the early stages of visual processing, such as when the brain is extracting surface-feature information like texture (CoS) or colour (V1). Future studies could consider the time course of these neural events and probe the importance of cross-modal (e.g., audition and haptic) information underpinning feature extraction for distinguishing real from fake objects.

We investigated the neural underpinnings of texture categorisation using exemplars that were previously learned either within modalities (visual training and visual test) or across modalities (tactile training and visual test). Previous models of learning suggest a decrease in activation in brain regions that are typically involved in cognitive control during task acquisition, but a concomitant increase in activation in brain regions associated with the representation of the acquired information. In our study, participants were required to learn to categorise fabrics of different textures as either natural or synthetic. Training occurred over several sessions, with each fabric presented either visually or through touch to a participant. Pre- and post-training tests, in which participants categorised visual images only of the fabrics, were conducted during a functional magnetic resonance imaging (fMRI) scan. Consistent with previous research on cognitive processes involved in task acquisition, we found that categorisation training was associated with a decrease in activation in brain regions associated with cognitive systems involved in learning, including the superior parietal cortex, dorsal anterior cingulate cortex (dACC), and the right dorsolateral prefrontal cortex (DLFC). Moreover, these decreases were independent of training modality. In contrast, we found greater activation to visual textures in a region within the left medial occipital cortex (MOC) following training. There was no overall evidence of an effect of training modality in the main analyses, with texture-specific regional changes associated with both within- (visual) and cross- (touch) modal training. However, further analyses suggested that, unlike categorisation performance following within-modal training, crossmodal training was associated with bilateral activation of the MOC. Our results support previous evidence for a multisensory representation of texture within early visual regions of the cortex and provide insight into how multisensory categories are formed in the brain.

Previous studies have suggested that discrete cross-sensory events could be incorrectly combined in the brain of older adults with a history of falls, possibly undermining motor and balance control. Based on previous findings that multisensory integration is modifiable with practice, even in an ageing population, we designed a serious game, named CityQuest, to train typical, everyday multisensory processes including sensori-motor control, spatial navigation, obstacle avoidance and balance control. Played over several sessions, this game was shown to improve these functions in older adults with and without a history of falls, depending on the specific condition of the game on which they were trained. Here, using voxel-based morphometry analysis of anatomical magnetic resonance imaging (MRI) data, we investigated structural changes in the brain of a smaller group of older adults from those who successfully completed this five-week intervention. A grey-matter (GM) volume increase in the precentral gyrus, and GM volume reduction in the inferior temporal and orbitofrontal gyri, was found for all participants. Changes in GM volume within regions of the cerebellum were differentially associated with fall-prone and healthy older adults. Furthermore, a greater GM volume increase in the precentral gyrus was observed in participants who performed the full CityQuest intervention relative to those required to avoid obstacles only. Our results support previous evidence that multisensory training can affect structural changes in the older brain and have implications for programmes designed for the successful rehabilitation of perceptual and cognitive functions.

Recent research has provided evidence suggesting a link between inefficient processing of multisensory information and incidence of falling in older adults. Specifically, Setti et al. (Exp Brain Res 209:375–384, 2011) reported that older adults with a history of falling were more susceptible than their healthy, age-matched counterparts to the sound-induced flash illusion. Here, we investigated whether balance control in fall-prone older adults was directly associated with multisensory integration by testing susceptibility to the illusion under two postural conditions: sitting and standing. Whilst standing, fall-prone older adults had a greater body sway than the age-matched healthy older adults and their body sway increased when presented with the audio–visual illusory but not the audio–visual congruent conditions. We also found an increase in susceptibility to the sound-induced flash illusion during standing relative to sitting for fall-prone older adults only. Importantly, no performance differences were found across groups in either the unisensory or non-illusory multisensory conditions across the two postures. These results suggest an important link between multisensory integration and balance control in older adults and have important implications for understanding why some older adults are prone to falling.

While aging can lead to significant declines in perceptual and cognitive function, the effects of age on multisensory integration, the process in which the brain combines information across the senses, are less clear. Recent reports suggest that older adults are susceptible to the sound-induced flash illusion (Shams et al., 2000) across a much wider range of temporal asynchronies than younger adults (Setti et al., 2011). To assess whether this cost for multisensory integration is a general phenomenon of combining asynchronous audiovisual input, we compared the time courses of two variants of the sound-induced flash illusion in young and older adults: the fission illusion, where one flash accompanied by two beeps appears as two flashes, and the fusion illusion, where two flashes accompanied by one beep appear as one flash. Twenty-five younger (18-30 years) and older (65+ years) adults were required to report whether they perceived one or two flashes, whilst ignoring irrelevant auditory beeps, in bimodal trials where auditory and visual stimuli were separated by one of six stimulus onset asynchronies (SOAs). There was a marked difference in the pattern of results for the two variants of the illusion. In conditions known to produce the fission illusion, older adults were significantly more susceptible to the illusion at longer SOAs compared to younger participants. In contrast, the performance of the younger and older groups was almost identical in conditions known to produce the fusion illusion. This surprising difference between sound-induced fission and fusion in older adults suggests dissociable age-related effects in multisensory integration, consistent with the idea that these illusions are mediated by distinct neural mechanisms.

From language to motor control, efficient integration of information from different sensory modalities is necessary for maintaining a coherent interaction with the environment. While a number of training studies have focused on training perceptual and cognitive function, only very few are specifically targeted at improving multisensory processing. Discrimination of temporal order or coincidence is a criterion used by the brain to determine whether cross-modal stimuli should be integrated or not. In this study we trained older adults to judge the temporal order of visual and auditory stimuli. We then tested whether the training had an effect in reducing susceptibility to a multisensory illusion, the sound induced flash illusion. Improvement in the temporal order judgement task was associated with a reduction in susceptibility to the illusion, particularly at longer Stimulus Onset Asynchronies, in line with a more efficient multisensory processing profile. The present findings set the ground for more broad training programs aimed at improving older adults׳ cognitive performance in domains in which efficient temporal integration across the senses is required.

Our understanding of human perception has developed significantly over the last 50 years, informed by research in neurophysiology, behavioural studies, psychophysics and neuroimaging. When the Department of Psychology at Trinity College Dublin was founded 50 years ago, teaching and research in perception was based on each sense in isolation, with a strong focus on vision. Recent research has revealed that perception in one sensory modality can be significantly modified by inputs from the other senses. Moreover, such cross-sensory interactions seem to occur much earlier in information processing than was historically assumed. Here we highlight some of the main studies that best demonstrate how research in multisensory perception has enhanced our understanding of how the human brain processes information from the external world. In particular, we focus on higher-level perceptual tasks such as object, face, and body perception, and the perception of socially meaningful information, such as emotion and attractiveness. We also explore how changes in multisensory processing occur throughout the lifespan. We argue that a multisensory approach provides us with a better insight into the functional properties of the perceptual brain.

It has previously been shown that older adults may be less efficient than younger adults at processing multisensory information, and that older adults with a history of falling may be less efficient than a healthy cohort when processing audio–visual stimuli (Setti et al., 2011). We investigated whether body stance has an effect on older adults’ ability to efficiently process multisensory information and also whether being presented with multisensory stimuli while standing may affect an individual’s balance. This experiment was performed by 44 participants, including both fall-prone older adults and a healthy control cohort. We tested their susceptibility to a sound-induced flash illusion (i.e., Shams et al., 2002), during both sitting and standing positions while measuring balance parameters using body-worn sensors. The results suggest that balance control in fall prone-adults was compromised relative to adults with no falls history, and this was particularly evident whilst they were presented with the auditory-flash illusion but not the non-illusory condition. Also, when the temporal window of the stimulus onset asynchrony was narrow (70 ms) fall-prone adults were more susceptible to the illusion during the standing position compared with their performance while seated, while the performance of older adults with no history of falling was unaffected by a change in position. These results suggest a link between efficient multisensory integration and balance control and have implications for interventions when fall-prone adults encounter complex multisensory information in their environment.