Despite good evidence for optimal audio-visual integration in stationary observers, few studies have considered the impact of self-movement on this process. When the head and/or eyes move, the integration of vision and hearing is complicated, as the sensory measurements begin in different coordinate frames. To successfully integrate these signals, they must first be transformed into the same coordinate frame. We propose that audio and visual motion cues are separately transformed using self-movement signals, before being integrated as body-centered cues to audio-visual motion. We tested this hypothesis using a psychophysical audio-visual integration task in which participants made left/right judgments of audio, visual, or audio-visual targets during self-generated yaw head rotations. Estimates of precision and bias from the audio and visual conditions were used to predict performance in the audio-visual conditions. We found that audio-visual performance was predicted well by models that suggested the transformation of cues into common coordinates but could not be explained by a model that did not rely on coordinate transformation before integration. We also found that precision specifically was better predicted by a model that accounted for shared noise arising from signals encoding head movement. Taken together, our findings suggest that motion perception in active observers is based on the integration of partially correlated body-centered signals.

ABSTRACT Everyday actions like moving the head, walking around and reaching out to grasp objects are typically self-controlled. This presents a problem when studying the signals encoding such actions because active self-movement is difficult to experimentally control. Available techniques demand repeatable trials, but each action is unique, making it difficult to measure fundamental properties like psychophysical thresholds. Here, we present a novel paradigm that can be used to recover both precision and bias of self-movement signals with minimal constraint on the participant. The paradigm takes care of a hidden source of external noise not previously accounted for in techniques that link display motion to self-movement in real time (e.g. virtual reality). We use head rotations as an example of self-movement, and show that the precision of the signals encoding head movement depends on whether they are being used to judge visual motion or auditory motion. We find perceived motion is slowed during head movement in both cases, indicating that the ‘non-image’ signals encoding active head rotation (motor commands, proprioception and vestibular cues) are biased to lower speeds and/or displacements. In a second experiment, we trained participants to rotate their heads at different rates and found that the precision of the head rotation signal rises proportionally with head speed (Weber’s Law). We discuss the findings in terms of the different motion cues used by vision and hearing, and the implications they have for Bayesian models of motion perception. NEW AND NOTEWORTHY We present a psychophysical technique for measuring the precision of signals encoding active self-movements. Using head movements, we show that: (1) precision declines when active head rotation is combined with auditory as opposed to visual motion; (2) precision rises with head speed (Weber’s Law); (3) perceived speed is lower during head movement. The findings may reflect the steps needed to convert different cues into common currencies, and challenge standard Bayesian models of motion perception.

Abstract Objective The use of various types of filtering facepiece class 3 (FFP3) mask have become commonplace since the Covid-19 outbreak. These have been evaluated in terms of efficacy regarding aerosol filtration but less emphasis has been placed on the acoustic effects of such masks and their consequences for clinical communication. Design A microphone 65 cm from a sound-producing Head and Torso Simulator (wearing the masks) was used to measure attenuation via a tone sweep. Predicted impact on speech reception in noise was assessed by weighting the attenuations of cochlear excitation patterns by the frequency importance function of the Speech Intelligibility Index. Study sample We evaluated acoustic attenuation properties of seven FFP3 masks and a Type IIR surgical mask (as a comparator). Results The Type IIR mask had the smallest impact on SNR (2.6 dB with visor). Most FFP3s with an addition of a visor (if not already face covering) impacted SNR by approximately 6 dB. The 3 M 6000 was significantly worse (15.8 dB). Conclusions Mouth-and-nose covering FFP3s masks had similar effects on SNR (≈6.2 dB with visor). The Tecmen TM-H2 had several advantages over other masks evaluated. It was reusable, allowed lipreading clues and the attenuation was similar to other FFP3s.

Knowing where our limbs are in space is crucial for a successful interaction with the external world. Joint position sense (JPS) relies on both cues from muscle spindles and joint mechanoreceptors, as well as the effort required to move. However, JPS may also rely on the perceived external force on the limb, such as the gravitational field. It is well known that the internal model of gravity plays a large role in perception and behaviour. Thus, we have explored whether direct vestibular-gravitational cues could influence JPS. Participants passively estimated the position of their hand while they were upright and therefore aligned with terrestrial gravity, or pitch-tilted 45° backwards from gravity. Overall participants overestimated the position of their hand in both upright and tilted postures; however, the proprioceptive bias was significantly reduced when participants were tilted. Our findings therefore suggest that the internal model of gravity may influence and update JPS in order to allow the organism to interact with the environment.

The sense of self is a complex phenomenon, comprising various sensations of bodily self-consciousness. Interestingly, the experience of possessing a body – ‘embodiment’ – and locating the body within space may be modulated by the Rubber Hand Illusion (RHI). Measures of the RHI include proprioceptive drift (PD), the extent to which the hand is mis-localised towards the rubber hand, and subjective questionnaires. Although these measures often correlate, research from the visual RHI suggests that they reflect separate underlying processes. We investigated whether increasing the duration of tactile stimulation would affect PD and questionnaires differently during the somatic RHI. Participants experienced 30 s, 2 min, or 5 min of synchronous or asynchronous tactile stimulation. Increasing duration affected only PD, with increased drift following 5 min vs 30 s of stimulation. Our findings suggest that PD and questionnaires are not proxies for one another, but reflect separate underlying processes of the somatic RHI.

During exposure to Virtual Reality (VR) a sensory conflict may be present, whereby the visual system signals that the user is moving in a certain direction with a certain acceleration, while the vestibular system signals that the user is stationary. In order to reduce this conflict, the brain may down-weight vestibular signals, which may in turn affect vestibular contributions to self-motion perception. Here we investigated whether vestibular perceptual sensitivity is affected by VR exposure. Participants' ability to detect artificial vestibular inputs was measured during optic flow or random motion stimuli on a VR head-mounted display. Sensitivity to vestibular signals was significantly reduced when optic flow stimuli were presented, but importantly this was only the case when both visual and vestibular cues conveyed information on the same plane of self-motion. Our results suggest that the brain dynamically adjusts the weight given to incoming sensory cues for self-motion in VR; however this is dependent on the congruency of visual and vestibular cues.

Human behaviour is a trade-off between exploitation of familiar resources and exploration of new ones. In a challenging environment-such as outer space-making the correct decision is vital. On Earth, gravity is always there, and is an important reference for behaviour. Thus, altered gravitational signals may affect behaviour control strategies. Here, we investigated whether changing the body's orientation to the gravitational vector would modulate the balance between routine and novel behaviour. Participants completed a random number generation task while upright or supine. We found decreased randomness when participants were supine. In particular, the degree of equiprobability of pairs of consecutive responses was reduced in the supine orientation. Online gravitational signals may shape the balance between exploitation and exploration, in favour of more stereotyped and routine responses.

The popularity of virtual reality (VR) has increased rapidly in recent years. While significant technological advancements are apparent, a troublesome problem with VR is that between 20% and 80% of users will experience unpleasant side effects such as nausea, disorientation, blurred vision and headaches-a malady known as Cybersickness. Cybersickness may be caused by a conflict between sensory signals for self-motion: while vision signals that the user is moving in a certain direction with certain acceleration, the vestibular organs provide no corroborating information. To resolve the sensory conflict, vestibular cues may be down-weighted leading to an alteration of how the brain interprets actual vestibular information. This may account for the frequently reported after-effects of VR exposure. Here, we investigated whether exposure to vection in VR modulates vestibular processing. We measured vestibular-evoked myogenic potentials (VEMPs) during brief immersion in a vection-inducing VR environment presented via head-mounted display. We found changes in VEMP asymmetry ratio, with a substantial increase in VEMP amplitude recorded on the left sternocleidomastoid muscle following just one minute of exposure to vection in VR. Our results suggest that exposure to vection in VR modulates vestibular processing, which may explain common after-effects of VR.

Egocentric frames of reference take the body as the point of origin of a spatial coordinate system. Bodies, however, are not points, but extended objects, with distinct parts that can move independently of one another. We recently developed a novel paradigm to probe the use of different body parts in simple spatial judgments, what we called the misalignment paradigm. In this study, we applied the misalignment paradigm in a perspective-taking task to investigate whether the weightings given to different body parts are shared across different spatial judgments involving different spatial axes. Participants saw birds-eye images of a person with their head rotated 45° relative to the torso. On each trial, a ball appeared and participants made judgments either of whether the ball was to the person's left or right, or whether the ball was in front of the person or behind them. By analysing the pattern of responses with respect to both head and torso, we quantified the contribution of each body part to the reference frames underlying each judgment. For both judgment types we found clear contributions of both head and torso, with more weight being given on average to the torso. Individual differences in the use of the two body parts were correlated across judgment types indicating the use of a shared set of weightings used across spatial axes and judgments. Moreover, retesting of participants several months later showed high stability of these weightings, suggesting that they are stable characteristics of people.

On Earth, we are continually exposed to gravity: sensory signals are constantly integrated to form an internal model of gravity. However, it is unclear whether this internal model is fixed to Earth's gravity or whether it can be applied to a new gravitational environment. Under terrestrial gravity, observers show a gravitational bias while judging the speed of falling versus rising objects, as they comply with the physical laws of gravity. We investigated whether this gravitational bias may be present when judging the speed of objects moving upwards or downwards in both virtual reality (VR)-simulated Earth gravity (9.81m/s(2)) and Mars gravity (3.71m/s(2)). Our results highlighted a gravitational bias in both Earth and Mars VR-simulated gravity: the speed of downwards movement was more precisely detected than the speed of upwards movement. Although the internal model of gravity has been built up under terrestrial gravity, it can quickly expand to novel non-terrestrial gravitational environments.

The body midline provides a basic reference for egocentric representation of external space. Clinical observations have suggested that vestibular information underpins egocentric representations. Here we aimed to clarify whether and how vestibular inputs contribute to egocentric representation in healthy volunteers. In a psychophysical task, participants were asked to judge whether visual stimuli were located to the left or to the right of their body midline. Artificial vestibular stimulation was applied to stimulate the vestibular organs. We found that artificial stimulation of the vestibular system biased body midline perception. Importantly, no effect was found on motor effector selection. We also ruled out additional explanations based on allocentric visual representations and on potential indirect effects caused by vestibular-driven movements of the eyes, head and body. Taken together our data suggest that vestibular information contributes to computation of egocentric representations by affecting the internal representation of the body midline.

Older adults struggle in dealing with changeable and uncertain environments across several cognitive domains. This has been attributed to difficulties in forming adequate task representations that help navigate uncertain environments. Here, we investigate how, in older adults, inadequate task representations impact on model-based reversal learning. We combined computational modeling and pupillometry during a novel model-based reversal learning task, which allowed us to isolate the relevance of task representations at feedback evaluation. We find that older adults overestimate the changeability of task states and consequently are less able to converge on unequivocal task representations through learning. Pupillometric measures and behavioral data show that these unreliable task representations in older adults manifest as a reduced ability to focus on feedback that is relevant for updating task representations, and as a reduced metacognitive awareness in the accuracy of their actions. Instead, the data suggested older adults' choice behavior was more consistent with a guidance by uninformative feedback properties such as outcome valence. Our study highlights that an inability to form adequate task representations may be a crucial factor underlying older adults' impaired model-based inference.

In the past decade, there has been a rapid advance in Virtual Reality (VR) technology. Key to the user's VR experience are multimodal interactions involving all senses. The human brain must integrate real-time vision, hearing, vestibular and proprioceptive inputs to produce the compelling and captivating feeling of immersion in a VR environment. A serious problem with VR is that users may develop symptoms similar to motion sickness, a malady called cybersickness. At present the underlying cause of cybersickness is not yet fully understood. Cybersickness may be due to a discrepancy between the sensory signals which provide information about the body's orientation and motion: in many VR applications, optic flow elicits an illusory sensation of motion which tells users that they are moving in a certain direction with certain acceleration. However, since users are not actually moving, their proprioceptive and vestibular organs provide no cues of self-motion. These conflicting signals may lead to sensory discrepancies and eventually cybersickness. Here we review the current literature to develop a conceptual scheme for understanding the neural mechanisms of cybersickness. We discuss an approach to cybersickness based on sensory cue integration, focusing on the dynamic re-weighting of visual and vestibular signals for self-motion.

Verticality plays a fundamental role in the arts, portraying concepts such as power, grandeur, or even morality; however, it is unclear whether people have an aesthetic preference for vertical stimuli. The perception of verticality occurs by integrating vestibular-gravitational input with proprioceptive signals about body posture. Thus, these signals may influence the preference for verticality. Here, we show that people have a genuine aesthetic preference for stimuli aligned with the vertical, and this preference depends on the position of the body relative to the gravitational direction. Observers rated the attractiveness of lines that varied in inclination. Perfectly vertical lines were judged to be more attractive than those inclined clockwise or anticlockwise only when participants held an upright posture. Critically, this preference was not present when their body was tilted away from the gravitational vertical. Our results showed that gravitational signals make a contribution to the perception of attractiveness of environmental objects.

Hemispatial neglect is a disorder of attention that commonly follows from damage to the right side of the brain. Patients with neglect show symptoms of lateralised inattention, failing to acknowledge or report information on the left side. Neglect is a poor prognostic indicator for general functional recovery from stroke, and is associated with a range of co-morbid conditions including denial or indifference to the brain injury, hemiplegia and visual field loss. Mild-to-moderate cases can be overshadowed by the more gross symptoms that accompany brain injury; however, assessment and diagnosis are relatively quick and simple. Current treatment guidelines suggest that patients should be taught compensatory strategies, but these are largely ineffective. Although recent research has identified more promising treatment approaches, investigations are still preliminary. Given the prevalence and debilitating nature of neglect, there is a clear need to raise awareness and understanding of the condition among carers and health professionals.