Effective rehabilitation is critical for individuals with motor deficits, such as following a stroke or brain injury. Although high doses of intensive motor rehabilitation can provide maximal benefit, several barriers can prevent attaining necessary therapy levels, including cost, physical access, clinical resource availability, and patient disengagement. Virtual Reality (VR) offers a potential solution to overcome these challenges and enable home-based, self-directed therapy. Here we present custom software that runs on the low-cost Meta Quest 2 headset to deliver immersive rehabilitation exercises. The system uses movement amplification to virtually reduce motor deficits, enabling successful task completion even in individuals with severe disabilities. This approach leverages reinforcement learning to maintain patient engagement during rehabilitation. The system capitalizes on recent advances in VR technology, including inside-out hand tracking, voice recognition, and a responsive virtual coach, to create a more accessible environment for users with no prior VR experience. We present the custom virtual rehabilitation implementation and test whether motor amplification leads to an increase in visually induced motion sickness experienced by individuals without disabilities. We find that motor amplification does not elevate reported motion sickness levels, suggesting that low-cost VR systems may enable increased access to individualized and guided motor rehabilitation.
The sense of agency is the perceptual experience of authoring our volitional actions. It is a valuable framework for assessing perceptions of control and the quality of interaction for users operating assistive devices. Intentional binding (IB) is a quantitative measure of the sense of agency where individuals perceive temporal compression between a volitional action and its sensory outcome. IB has been investigated during simple button press actions, but little work extends findings to gross whole-limb movements that are more relevant to assistive or rehabilitative robotics. This study develops and examines the feasibility of a novel, high-resolution approach for assessing IB using a forced two-choice discrimination paradigm. We designed a target acquisition task that required movement of the whole upper limb. Eight participants discriminated time durations between pairs of auditory tones coinciding with paired active and passive movements executed on a Kinarm. Participants reported which movement had a shorter time interval between tones while we varied time delays across movements and trials. We fit psychometric curves to participant responses and achieve a high-resolution estimate of IB biases while also providing a new slope dimension of analysis for IB experiences. The median bias across participants was -36.2ms (IQR: 145.1). We measured IB biases in whole limb movements with high resolution using this method and share its implications for assistive device design.
Stroke is a leading cause of disability worldwide. Following a stroke, high doses of intensive motor rehabilitation provide maximal benefit. Barriers to realizing recommended levels of rehabilitation therapy include cost, physical access, and clinical resource availability. Virtual Reality (VR) offers a potential solution to overcome these challenges and enable home-based, self-directed therapy. Here we present a low-cost system utilizing the off-the-shelf Meta Quest 2 headset running custom software to deliver immersive rehabilitation exercises. The system employs movement amplification to partially reduce motor deficits and enable more engaging task completion. It leverages recent advances in VR technology, including hand tracking, voice recognition, and an AI-driven virtual coach, to create a more accessible environment for users with no prior VR experience. We tested the system in 60 able-bodied participants to assess feasibility for eventual clinical use. Participants completed three VR tasks with 30 participants receiving motor amplification. We measured motion sickness levels and attitudes towards the technology, while generating a set of normative motor performance data. Results showed no significant effect of amplification on motor performance, indicating that the small amplification factors applied in this healthy sample did not markedly impact task outcomes. User attitudes towards VR improved after the experience and minimal motion sickness was reported. This study suggests that low-cost VR systems may serve as a feasible option to increase post-stroke motor rehabilitation. Further study is needed to test the system in older adults and in stroke patients to determine feasibility in a more representative clinical setting. ### Competing Interest Statement The authors have declared no competing interest. ### Funding Statement This study was funded by ResearchNS under the New Health Investigator Grant program. ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: The Research Ethics Board of Acadia University (Wolfville, NS, Canada) gave ethical approval for this work (REB File # 22-07). I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes All data produced in the present study are available upon reasonable request to the authors and will be available online in the future.
Sensorimotor adaptation is supported by at least two parallel learning systems: an intentionally controlled explicit strategy, and an involuntary implicit learning system. To investigate the error sources driving these two systems, past work focused on constrained reaches or finger movements in laboratory environments has shown subconscious learning systems to be driven in part by sensory prediction error (SPE), i.e., the mismatch between the realized and expected outcome of an action. We designed a ball rolling task to explore whether SPEs can drive implicit motor adaptation during complex whole-body movements that impart physical motion on external objects. After applying a visual shift, participants rapidly adapted their rolling angles to reduce the error between the ball and target. We removed all visual feedback and told participants to aim their throw directly toward the primary target, revealing an unintentional 5.06° implicit adjustment to reach angles that decayed over time. To determine whether this implicit adaptation was driven by SPE, we gave participants a second aiming target that would ‘solve’ the visual shift, as in Mazzoni and Krakauer (2006). Remarkably, after rapidly reducing ball rolling error to zero (due to enhancements in strategic aiming), the additional aiming target caused rolling angles to deviate beyond the primary target by 3.15°. This involuntary overcompensation, which worsened task performance, is a hallmark of SPE-driven implicit learning. These results show that SPE-driven implicit processes, previously observed within simplified finger or planar reaching movements, actively contribute to motor adaptation in more complex naturalistic skill-based tasks. New and Noteworthy Implicit and explicit learning systems have been detected using simple, constrained movements inside the laboratory. How these systems impact movements during complex whole-body, skill-based tasks has not been established. Here we demonstrate that sensory prediction errors significantly impact how a person updates their movements, replicating findings from the laboratory in an unconstrained ball-rolling task. This real-world validation is an important step towards explaining how subconscious learning helps humans execute common motor skills in dynamic environments.
Changes in alpha band activity (8-12 Hz) indicate the downregulation of brain regions during cognitive tasks, reflecting real-time cognitive load. Despite this, its feasibility to be used in a more dynamic environment with ongoing motor corrections has not been studied. This research used electroencephalography (EEG) to explore how different brain regions are engaged during a simple grasp and lift task where unexpected changes to the object's weight or surface friction are introduced. The results suggest that alpha activity changes related to motor error correction occur only in motor-related areas (i.e. central areas) but not in error processing areas (i.e., frontoparietal network) during unexpected weight changes. This suggests that oscillations over motor areas reflect the reduction of motor drive related to motor error correction, thus, being a potential cortical electrophysiological biomarker for the process and not solely as a proxy for cognitive demands. This observation is particularly relevant in scenarios where these signals are used to evaluate high cognitive demands co-occurring with high levels of motor errors and corrections, such as prosthesis use. The establishment of electrophysiological biomarkers of mental resource allocation during movement and cognition can help identify indicators of mental workload and motor drive, which may be useful for improving brain-machine interfaces. NEW & NOTEWORTHY We demonstrated that alpha suppression, an EEG phenomenon with high temporal resolution, occurs over the primary sensorimotor area during error correction during lift movements. Interpretations of alpha activity are often attributed to high cognitive demands, thus recognizing that it is also influenced by motor processes is important in situations where cognitive demands are paired with movement errors. This could further have application as a biomarker for error correction in human-machine interfaces, such as neuroprostheses.
Connectivity in the frontoparietal network of the brain has been established as a marker of neural processes related to working memory. This work evaluated a power-based correlation method to investigate the frontoparietal connectivity of 13 participants during a working memory task across the theta, alpha, and beta frequencies. Higher connectivity of electrodes was found within functional regions compared to connectivity outside functional regions of the brain across all frequencies, suggesting that the spatial resolution of this method is sufficient to assess connectivity at a functional level. The primary finding was that frontoparietal connectivity in the alpha frequency band was higher compared to the other two frequencies. These results show that the method used here can display the functional role of alpha connectivity against other frequencies, further supporting the role of the alpha band as a neurophysiological marker of information processing in the brain. This method may be useful to obtain physiologically relevant features of working memory to improve EEG human-machine interfaces.
When a person makes a movement, a motor error is typically observed that then drives motor planning corrections on subsequent movements. This error correction, quantified as a trial-by-trial adaptation rate, provides insight into how the nervous system is operating, particularly regarding how much confidence a person places in different sources of information such as sensory feedback or motor command reproducibility. Traditional analysis has required carefully controlled laboratory conditions such as the application of perturbations or error clamping, limiting the usefulness of motor analysis in clinical and everyday environments. Here we focus on error adaptation during unperturbed and naturalistic movements. With increasing motor noise, we show that the conventional estimation of trial-by-trial adaptation increases, a counterintuitive finding that is the consequence of systematic bias in the estimate due to noise masking the learner’s intention. We present an analytic solution relying on stochastic signal processing to reduce this effect of noise, producing an estimate of motor adaptation with reduced bias. The result is an improved estimate of trial-by-trial adaptation in a human learner compared to conventional methods. We demonstrate the effectiveness of the new method in analyzing simulated and empirical movement data under different noise conditions.
During goal-directed movements, the magnitude of error correction by a person on a subsequent movement provides important insight into a person’s motor learning dynamics. Observed differences in trial-by-trial adaptation rates might indicate different relative weighting placed on the various sources of information that inform a movement, e.g. sensory feedback, control predictions, or internal model expectations. Measuring this trial-by-trial adaptation rate is not straightforward, however, since externally observed data are masked by noise from several sources and influenced by inaccessible internal processes. Adaptation to perturbation has been used to measure error adaptation as the introduced external disturbance is sufficiently large to overshadow other noise sources. However, perturbation analysis is difficult to implement in real-world scenarios, requires a large number of movement trials to accommodate infrequent perturbations, and the paradigm itself might affect the movement dynamics being observed. Here we focus on error adaptation during unperturbed and naturalistic movements. With increasing motor noise, the conventional estimation of trial-by-trial adaptation increases, a counterintuitive finding that is the consequence of systematic bias in the estimate due to noise masking the learner’s intention. We present an analytic solution relying on stochastic signal processing to reduce this effect of noise, producing an estimate of motor adaptation with reduced bias. The result is an improved estimate of trial-by-trial adaptation in a human learner compared to conventional methods. We demonstrate the effectiveness of the new method in analyzing simulated and empirical movement data under different noise conditions. The analytic approach is applicable across different types of movements in varied contexts and should replace the regression analysis method in future motor analysis studies.Author Summary When a person makes a movement, a motor error is typically observed that then drives motor planning corrections on subsequent movements. This error correction provides insight into how the nervous system is operating, particularly in regard to how much confidence a person places in different sources of information such as sensory feedback or motor command reproducibility. Traditional analysis of movement has required carefully controlled laboratory conditions, limiting the usefulness of motor analysis in clinical and everyday environments. Here we present a new computational method that can be accurately applied to typical movements. Counterintuitive findings of the established approach are corrected by the proposed method. This method will provide a common framework for researchers to analyze movements while extending dynamic motor adaptation analysis capabilities to clinical and non-laboratory settings.### Competing Interest StatementThe authors have declared no competing interest.
Human Machine Interfaces (HMIs) can provide critical support and improve daily task functionality for prosthesis users or social interaction for patients with locked-in syndrome using an assistive communication device. One goal in the development of sophisticated HMIs is to reduce the cognitive load (CL) they place on the user to promote the use of the technology. Electroencephalogram (EEG)-derived measures collected with wired wet-electrode systems have been used to assess CL in laboratory environments and have demonstrated acceptable test-retest reliability. Assessment of CL during real-world unconstrained HMI operation, however, requires the use of a wireless dry-electrode EEG system which provides easier electrode application and untethered movement. However, the test-retest reliability of wireless dry-electrode systems to quantify CL has not been explored. Ensuring the consistent capture of CL-related signals across multiple sessions is critical if these devices are to be used to assess how improvements in HMIs affect CL. Therefore, the current study used a wireless dry-electrode EEG system to compare Evoked Response Potential (ERP) features of a simple auditory oddball task to measure CL during two separate testing sessions a week apart. ERPs of 11 subjects were recorded while participants performed a virtual task at two difficulty levels. A significant correlation was found between the P300 component of the ERPs and subjective ratings of CL during both testing sessions. Furthermore, there was a statistically significant test-retest reliability for this same ERP feature and similar signal-to-noise ratios (SNRs) across sessions.Clinical Relevance- This is an initial step in validating wireless dry-electrode EEG systems to assess cognitive load across multiple sessions. The evidence presented is critical if dry-wireless EEG systems are to be used to identify aspects of HMIs that reduce CL in clinical and real-life environments. Assessing CL in unconstrained environments can better inform clinicians and technology developers in their design of future HMIs.
A correction to this article has been published and is linked from the HTML and PDF versions of this paper. The error has not been fixed in the paper.
Research on human motor adaptation has often focused on how people adapt to self-generated or externally-influenced errors. Trial-by-trial adaptation is a person's response to self-generated errors. Externally-influenced errors applied as catch-trial perturbations are used to calculate a person's perturbation adaptation rate. Although these adaptation rates are sometimes compared to one another, we show through simulation and empirical data that the two metrics are distinct. We demonstrate that the trial-by-trial adaptation rate, often calculated as a coefficient in a linear regression, is biased under typical conditions. We tested 12 able-bodied subjects moving a cursor on a screen using a computer mouse. Statistically different adaptation rates arise when sub-sets of trials from different phases of learning are analyzed from within a sequence of movement results. We propose a new approach to identify when a person's learning has stabilized in order to identify steady-state movement trials from which to calculate a more reliable trial-by-trial adaptation rate. Using a Bayesian model of human movement, we show that this analysis approach is more consistent and provides a more confident estimate than alternative approaches. Constraining analyses to steady-state conditions will allow researchers to better decouple the multiple concurrent learning processes that occur while a person makes goal-directed movements. Streamlining this analysis may help broaden the impact of motor adaptation studies, perhaps even enhancing their clinical usefulness.
Advanced neural interfaces show promise in making prosthetic limbs more biomimetic and ultimately more intuitive and useful for patients. However, approaches to assess these emerging technologies are limited in scope and the insight they provide. When outfitting a prosthesis with a feedback system, such as a peripheral nerve interface, it would be helpful to quantify its physiological correspondence, i.e. how well the prosthesis feedback mimics the perceived feedback in an intact limb. Here we present an approach to quantify this aspect of feedback quality using the crossmodal congruency effect (CCE) task. We show that CCE scores are sensitive to feedback modality, an important characteristic for assessment purposes, but are confounded by the spatial separation between the expected and perceived location of a stimulus. Using data collected from 60 able-bodied participants trained to control a bypass prosthesis, we present a model that results in adjusted-CCE scores that are unaffected by percept misalignment which may result from imprecise neural stimulation. The adjusted-CCE score serves as a proxy for a feedback modality's physiological correspondence or 'naturalness'. This quantification approach gives researchers a tool to assess an aspect of emerging augmented feedback systems that is not measurable with current motor assessments.
Current motor assessment tools can provide numerical indicators of performance but do not provide actionable information to target further improvement in rehabilitation interventions. Psychophysics-based outcome measures show promise to provide more useful information in the laboratory environment but have been limited in clinical implementation. Here we present a constrained-time task to assess paced and non-rhythmic movements. The task's output metrics include trial-by-trial adaptation rate and the just noticeable difference of a perturbation. We show that the task's metrics are reliable (i.e. high test-retest reliability) and are responsive to changes in feedback type and experience. We also discuss the task's versatility to be used for other types of movements including grasping. The consistent, sensitive and flexible time-constrained movement task we present provides a foundation from which to develop advanced outcome measures for prosthesis users and for other rehabilitation contexts.
The incorporation of feedback into a person’s body schema is well established. The crossmodal congruency task (CCT) is used to objectively quantify incorporation without being susceptible to experimenter biases. This visual-tactile interference task is used to calculate the crossmodal congruency effect (CCE) score as a difference in response time between incongruent and congruent trials. Here we show that this metric is susceptible to a learning effect that causes attenuation of the CCE score due to repeated task exposure sessions. We demonstrate that this learning effect is persistent, even after a 6 month hiatus in testing. Two mitigation strategies are proposed: 1. Only use CCE scores that are taken after learning has stabilized, or 2. Use a modified CCT protocol that decreases the task exposure time. We show that the modified and shortened CCT protocol, which may be required to meet time or logistical constraints in laboratory or clinical settings, reduced the impact of the learning effect on CCT results. Importantly, the CCE scores from the modified protocol were not significantly more variable than results obtained with the original protocol. This study highlights the importance of considering exposure time to the CCT when designing experiments and suggests two mitigation strategies to improve the utility of this psychophysical assessment.
A variety of factors affect the performance of a person using a myoelectric prosthesis, including increased control noise, reduced sensory feedback, and muscle fatigue. Many studies use able-bodied subjects to control a myoelectric prosthesis using a bypass socket in order to make comparisons to movements made with intact limbs. Depending on the goals of the study, this approach can also allow for greater subject numbers and more statistical power in the analysis of the results. As we develop assessment tools and techniques to evaluate how peripheral nerve interfaces impact prosthesis incorporation, involving normally limbed subjects in the studies becomes challenging. We have designed a novel bypass prosthesis to allow for the assessment of prosthesis incorporation in able-bodied subjects. Incorporation of a prosthetic hand worn by a normally limbed subject requires that the prosthesis is a convincing, functional extension of their own body. We present the design and development of the bypass prosthesis with special attention to mounting position and angle of the prosthetic hand, the quality of the control system and the responsiveness of the feedback. The bypass prosthesis has been fitted with a myoelectrically-controlled hand that has been instrumented to measure the forces applied to the thumb, index, and middle fingers. The prosthetic hand was mounted on the bypass socket such that it is the same length as the subject's intact limb but at a medial rotation angle of 20° to prevent visual occlusion of the prosthetic hand. Force feedback is provided in the form of electrical stimulation, vibration, or force applied to the intact limb with milliseconds of delay. Preliminary data results from a cross-modal congruency task are included showing evidence of prosthesis incorporation in able-bodied subjects. This bypass will allow able-bodied subjects to participate in research studies that require the use of a prosthetic limb while also allowing the subjects to sense that the prosthesis is an extension of the body.
The incorporation of a tool into a person9s body representation is well established. Quantitative measures play an important role in assessing tool incorporation levels for tool use paradigms. The crossmodal congruency effect (CCE) is used to quantify tool incorporation without being susceptible to experimenter biases. The crossmodal congruency task is a visual-tactile interference task that is used to calculate the CCE score as a difference in response time for incongruent and congruent trials. Here we show that this metric is susceptible to a learning effect that causes attenuation of the CCE score due to repeated task exposure sessions. This study investigated the conditions under which CCE scores attenuated due to task overexposure and tested if a modified version of the crossmodal congruency task could reduce the learning effect. Our work also sought to examine if the attenuated CCE scores returned to baseline values after a period of time. Thirty subjects were tested up to a maximum of ten times and four of these subjects were retested after a four month delay period. We show that CCE score reduced as early as the second exposure with a 14.5% drop between first and second exposures (p=0.027). Importantly, we found evidence that a modified version of the crossmodal congruency task, in which each exposure was reduced from eight to four test blocks, reduced the drop between first and second exposure from an average of 14.5% to 6.5% without significantly increasing variability of the measurement. Additionally, we found that three out of four subjects that were retested after a four month period returned to near-baseline CCE scores. This study highlights the importance of limiting exposure to the crossmodal congruency task, and proposes a modified approach to improve the use of this psychophysical assessment in the future.
Octopus localize their drill holes on prey but it remains unclear how this is accomplished. Asymmetrical prey such as snails and crabs would provide cues to the octopus but the nominally symmetrical cowry presents a unique drill hole localization scenario. To investigate how octopus drill cowries we collected their shells from octopus middens in Bonaire and surveyed drilled specimens in shell collections. Cowry drill holes were predominantly localized to the ventral-posterior-lateral region of the shell, the attachment point of the columellar muscle. Cowries with multiple drill holes suggest a trial-and-error learning process leading to localization. Further studies with naive octopuses are proposed to determine the role of learning in this feeding behavior.
As part of the Robobee project, we have modified a coaxial helicopter to operate using a discrete time map-based neuronal network for the control of heading, altitude, yaw, and odometry. Two concepts are presented: 1. A model for the integration of sensory data into the neural network. 2. A function for transferring the instantaneous spike frequency of motor neurons to a pulse width modulated signal required to drive motors and other types of actuators. The helicopter is provided with a flight vector and distance to emulate the information conveyed by the honeybee's waggle dance. This platform allows for the testing of proposed networks for adaptive navigation in an effort to simulate honeybee foraging on a flying robot.
We present a method to use the commercially available LEGO Mindstorms NXT robotics platform to test systems level neuroscience hypotheses. The first step of the method is to develop a nervous system simulation of specific reflexive behaviors of an appropriate model organism; here we use the American Lobster. Exteroceptive reflexes mediated by decussating (crossing) neural connections can explain an animal's taxis towards or away from a stimulus as described by Braitenberg and are particularly well suited for investigation using the NXT platform.1 The nervous system simulation is programmed using LabVIEW software on the LEGO Mindstorms platform. Once the nervous system is tuned properly, behavioral experiments are run on the robot and on the animal under identical environmental conditions. By controlling the sensory milieu experienced by the specimens, differences in behavioral outputs can be observed. These differences may point to specific deficiencies in the nervous system model and serve to inform the iteration of the model for the particular behavior under study. This method allows for the experimental manipulation of electronic nervous systems and serves as a way to explore neuroscience hypotheses specifically regarding the neurophysiological basis of simple innate reflexive behaviors. The LEGO Mindstorms NXT kit provides an affordable and efficient platform on which to test preliminary biomimetic robot control schemes. The approach is also well suited for the high school classroom to serve as the foundation for a hands-on inquiry-based biorobotics curriculum.