Reinforcement learning is a fundamental process through which humans and other animals attain rewards for themselves. However, to act prosocially, we must also learn how our choices reward others. The ventromedial prefrontal cortex has been independently linked to reinforcement learning and prosocial behaviour, yet its causal impact on prosocial reinforcement learning and the roles of its multiple subregions remain unknown.Here, a large group of adults with rare focal ventromedial prefrontal cortex damage (n = 28) and two carefully age- and gender-matched control groups (lesions elsewhere, n = 21; healthy controls, n = 124) completed a reinforcement learning task where they learnt to win rewards for another person (prosocial), for themselves (self) or in a control condition where participants saw points, but they were not translated into rewards for either individual (no one, control condition) on separate trials. A novel computational model incorporating separate learning rates for positive and negative prediction errors best explained behaviour in all groups.Importantly, compared to both control groups, patients with ventromedial prefrontal cortex damage were less accurate and had lower learning rates from positive prediction errors when rewarding another person relative to when no one benefitted, and higher learning rates for negative prediction errors when learning for others relative to self. Unlike controls, ventromedial prefrontal cortex lesion patients also showed a reduced self-benefitting advantage. They were equally accurate and learnt at a similar rate from positive prediction errors for self and neither individual. Strikingly, voxel-based lesion-symptom mapping revealed that damage to subgenual anterior cingulate cortex and anterior cingulate cortex gyrus specifically disrupted prosocial reinforcement learning.These findings highlight the importance of ventromedial prefrontal cortex integrity for multiple aspects of reinforcement learning, with damage to subgenual anterior cingulate cortex and anterior cingulate cortex gyrus critical in learning to reward others. Using a novel computational neurology approach in a cohort of patients with rare focal ventromedial prefrontal cortex damage, Gueguen et al. show that this brain region is necessary for learning to reward others, with different subregions playing distinct roles. The results have implications for the neurobiological basis of social behaviour.

Ventromedial prefrontal cortex (vmPFC) is vital for decision-making. Functional neuroimaging links vmPFC to processing rewards and effort, while parallel work suggests vmPFC involvement in prosocial behaviour. However, the necessity of vmPFC for these functions is unknown. Patients with rare focal vmPFC lesions (n = 25), patients with lesions elsewhere (n = 15) and healthy controls (n = 40) chose between rest and exerting effort to earn rewards for themselves or another person. vmPFC damage decreased prosociality across behavioural and computational measures. vmPFC patients earned less, discounted rewards by effort more, and exerted less force when another person benefited, compared to both control groups. Voxel-based lesion mapping revealed dissociations between vmPFC subregions. While medial damage led to antisocial behaviour, lateral damage increased prosocial behaviour relative to patients with damage elsewhere. vmPFC patients also showed reduced effort sensitivity overall, but reward sensitivity was limited to specific subregions. These results reveal multiple causal contributions of vmPFC to prosocial behaviour, effort and reward. By combining advanced mathematical modelling with data from a rare sample of patients with brain damage, the authors show that a specific part of the brain in the ventromedial prefrontal cortex is associated with putting in effort to help other people.

Apathy is recognized to be a common, disabling syndrome that occurs across a range of psychiatric and neurological conditions, including Parkinson’s disease. It can have a significant impact on quality of life, both for people affected and those around them. Currently, there are no established, evidence-based treatments for this debilitating syndrome. Assessment and treatment have been complicated by overlaps with depression and anhedonia, as well as a lack of understanding of the underlying mechanisms. Emerging lines of evidence conceptualize apathy as a reduction of motivation associated with disordered effort-based decision-making and dysfunction of distinct neural circuitry between the basal ganglia and medial prefrontal cortex. Here, we introduce a novel cognitive-behavioral framework that can inform a clinician’s conceptualization and treatment of apathy, using cognitive-behavioral therapy (CBT) techniques. We focus on people with Parkinson’s disease in our model, but our approach is transdiagnostic and can be applied to other conditions. It considers both individual targets for therapy as well as maintenance and intervention at a systemic level. The generalizability and parsimony of the framework provides a structured assessment and formulation of apathy, while also allowing clinicians to remain sensitive to other neuropsychiatric symptoms that can occur alongside apathy, such as depression and anxiety.

Abstract Idiopathic rapid eye movement sleep behaviour disorder (iRBD) has now been established as an important marker of the prodromal stage of Parkinson’s disease and related synucleinopathies. However, although dopamine transporter single photon emission computed tomography (SPECT) has been used to demonstrate the presence of nigro-striatal deficit in iRBD, quantifiable correlates of this are currently lacking. Sensitivity to rewarding stimuli is reduced in some people with Parkinson’s disease, potentially contributing to aspects of the neuropsychiatric phenotype in these individuals. Furthermore, a role for dopaminergic degeneration is suggested by the fact that reward insensitivity can be improved by dopaminergic medications. Patients with iRBD present a unique opportunity to study the relationship between reward sensitivity and early dopaminergic deficit in the unmedicated state. Here, we investigate whether a non-invasive, objective measure of reward sensitivity might be a marker of dopaminergic status in prodromal Parkinson’s disease by comparing with SPECT/CT measurement of dopaminergic loss in the basal ganglia. Striatal dopaminergic deficits in iRBD are associated with progression to Parkinsonian disorders. Therefore, identification of a clinically measurable correlate of this degenerative process might provide a basis for the development of novel risk stratification tools. Using a recently developed incentivized eye-tracking task, we quantified reward sensitivity in a cohort of 41 patients with iRBD and compared this with data from 40 patients with Parkinson’s disease and 41 healthy controls. Patients with iRBD also underwent neuroimaging with dopamine transporter SPECT/CT. Overall, reward sensitivity, indexed by pupillary response to monetary incentives, was reduced in iRBD cases compared with controls and was not significantly different to that in patients with Parkinson’s disease. However, in iRBD patients with normal dopamine transporter SPECT/CT imaging, reward sensitivity was not significantly different from healthy controls. Across all iRBD cases, a positive association was observed between reward sensitivity and dopaminergic SPECT/CT signal in the putamen. These findings demonstrate a direct relationship between dopaminergic deficit and reward sensitivity in patients with iRBD and suggest that measurement of pupillary responses could be of value in models of risk stratification and disease progression in these individuals.

Microsurgery is a particularly impactful yet challenging form of surgery. Robot assisted microsurgery has the potential to improve surgical dexterity and enable precise operation on such small scales in ways not previously possible. Intraocular microsurgery is a particularly challenging domain in part due to the lack of dexterity that is achievable with rigid instruments inserted through the eye. In this work, we present a new design for a millimeter-scale, dexterous wrist intended for microsurgery applications. The wrist is created via a state-of-the-art two-photon-polymerization (2PP) microfabrication technique, enabling the wrist to be constructed of flexible material with complex internal geometries and critical features at the micron-scale. The wrist features a square cross section with side length of 1.25 mm and total length of 3.75 mm. The wrist has three tendons routed down its length which, when actuated by small-scale linear actuators, enable bending in any plane. We present an integrated gripper actuated by a fourth tendon routed down the center of the robot. We evaluate the wrist and gripper by characterizing its bend-angle. We achieve more than 90 degrees bending in both axes. We demonstrate out of plane bending as well as the robot's ability to grip while actuated. Our integrated gripper/tendon-driven continuum robot design and meso-scale assembly techniques have the potential to enable small-scale wrists with more dexterity than has been previously demonstrated. Such a wrist could improve surgeon capabilities during teleoperation with the potential to improve patient outcomes in a variety of surgical applications, including intraocular surgery.

Understanding near-surface wind variability is crucial to support wind power penetration on national electrical grids. High-resolution numerical simulations are often proposed as the best solution to study the fluctuation of wind resources. We compare Weather Research and Forecasting (WRF) and Modern-Era Retrospective Analysis for Research and Applications version 2 (MERRA-2) bias-corrected wind speeds at hub height at different spatial resolutions and transform them to wind power production using a logistic power curve fitted to wind power measurements; the comparisons are based on error statistics and time series spectral analysis. The results show that numerical models reproduce observed wind speeds with correlations higher than 0.9 for WRF and 0.8 for MERRA-2. Moreover, annual observed wind power is reproduced with a maximum difference from observations of 0.011. However, each resolution reproduces the magnitudes of high-resolution periodicities differently so that there is a clear relationship between grid size and signal variance at high frequencies, as variance is indirectly proportional to frequency. This relationship is expected for wind speed, but based on results, it can be associated also for capacity factor sampled at hourly intervals. Therefore, the main benefit of high spatial resolution lies in the added variance in frequencies at sub-daily time scales. The study of the added value of high-resolution simulations in this region contributes to current efforts to develop reliable forecasting tools and strategies to support the development of wind power as a reliable energy source.

Apathy and impulsivity are expressed in a wide range of neuropsychiatric disorders, and, to a less severe extent, in healthy people too. Although traditionally considered to be opposite extremes of a single motivational spectrum, recent epidemiological questionnaire-based data suggest that both traits can in fact co-exist within the same individual. Here, we sought to investigate the relationship between these constructs in healthy people within a controlled task environment that examines the ability to make a decision under temporal uncertainty and measures the vigour of the response. Sixty participants performed a new version of the Traffic Light Task and completed self-report questionnaire measures of apathy and impulsivity. The task required individuals to make rapid decision-making for time-sensitive reward by squeezing a hand-held dynamometer as quickly as possible after a predictable event occurred (a traffic light turning green). Although apathy and impulsivity were positively correlated in questionnaire assessments, the two traits were associated with distinct behavioural signatures on the task. Impulsivity was expressed as an inflexible tendency to generate rapid anticipatory responses, regardless of cost-benefit information. Apathy, on the other hand, was associated with a blunted effect of reward on response vigour. These findings reveal how apathy and impulsivity are related to distinct dimensions of goal-directed behaviour, explaining how these traits might co-exist in the same individuals.

IntroductionClinical apathy is a poorly understood neuropsychiatric syndrome characterised by a sig- nificant decrease in goal-directed, motivated behaviour. It occurs in ~30% of patients with cerebrovas- cular small vessel disease (SVD). With the aim of improving our mechanistic understanding of apathy, we conducted a multimodal investigation combining validated behavioural paradigms and magnetic resonance imaging (MRI) techniques.Methods83 patients with MRI evidence of SVD were recruited from the Oxford Vascular Study (OXVASC) and Oxford neurology clinics. They were investigated using a novel effort-based decision making task and the Apathy Evaluation Scale (AES). Structural and diffusion weighted MRI was conducted to measure white matter lesion load (WMLL) and tract integrity, indexed by Fractional anisotropy (FA).ResultsPatients with apathy demonstrated a significant reduction in motivated behaviour and were sig- nificantly less incentivised by low levels of reward. Diffusion weighted imaging demonstrated that apathy was characterised by focal changes to limbic association tracts, including the uncinate fasciculus and cingulum bundle, as well as fronto-striatal white matter tracts. Importantly, global measures of disease severity did not independently associate with apathy.DiscussionReduced incentivisation by low reward characterised apathy in SVD, as previously reported in Parkinson’s disease, suggesting a common mechanism underlying apathy across neurological diseases. The association of apathy with focal white matter tract changes is consistent with disruption to key frontal and fronto-striatal circuits which have been previously implicated in effort-based decision-making for rewards.youssuf.saleh@ndcn.ox.ac.uk

In this study, we use a k-mean clustering approach to investigate the weather patterns responsible for extreme wind speed events throughout Mexico using 40 years of the ERA-5 atmospheric reanalysis. Generally, we find a large geographical split between the weather patterns that generate the strongest winds across the country. The highest wind power production periods therefore occur at different times in different regions across the country. In the South, these are associated with cold surge events, where an anticyclone is present in the Gulf of Mexico resulting in a strong Northerly flow across the Isthmus of Tehuantepec. In the North-East, Easterly trade winds are responsible for the strongest wind events, whereas in the North-West, it is the proximity of the North Pacific High. However, the weakest winds and lowest power production periods occur at the same times for all stations with the exception of Baja California Sur, meaning that low wind power production may be unavoidable at these times. The El Nino Southern Oscillation is found to influence wind speeds at some locations across Mexico at sub-seasonal time-scales. We report that statistically stronger wind speeds are observed during the Summer during El Nino months than during La Nina months for both sites in Chiapas and Oaxaca.

Apathy is a common, disabling neuropsychiatric syndrome that occurs across many brain disorders and may be associated with diminished motivation in behavioural, cognitive, emotional and social domains. Assessment is complicated by the variability of symptoms across apathy domains and self‐report from patients, which can be misleading due to their lack of insight. Independent evaluation by clinicians also has limitations though if it has to be performed with limited time. Caregiver reports are a viable alternative, but current assessments for them either do not distinguish between different apathy domains or are interview‐based and take long to administer. In this study, we developed a brief caregiver questionnaire version of the recently developed Apathy Motivation Index (AMI), which is a self‐report tool. We confirmed three apathy factors in this new caregiver measure (AMI‐CG) that were also present in the AMI: Behavioural Activation, Emotional Sensitivity and Social Motivation. Furthermore, we validated the scores against more extensive caregiver interviews using the established Lillle apathy rating scale as well as patient self‐reports of apathy, measures of depression, anhedonia, cognition, activities of daily living and caregiver burden across four different neurological conditions: Parkinson's disease, Alzheimer's disease, subjective cognitive impairment and limbic encephalitis. The AMI‐CG showed good internal reliability, external validity and diagnostic accuracy. It also uncovered cases of social apathy overlooked by traditional instruments. Crucially, patients who under‐rated their apathy compared to informants were more likely to have difficulties performing everyday activities and to be a greater burden to caregivers. The findings provide evidence for a multidimensional conceptualization of apathy and an instrument for efficient detection of apathy based on caregiver reports for use in clinical practice.

Social cohesion relies on prosociality in increasingly ageing populations. Helping others requires effort, yet how willing people are to exert effort to benefit ourselves and other people, and whether such behaviours shift across the lifespan, is poorly understood. Using computational modelling we tested the willingness to exert effort into self or other benefitting acts in younger (age 18-36) and older adults (55-84, total n=187). Participants chose whether to work and exert effort, (between 30-70% of maximum grip strength) for rewards (2-10 credits) accrued for themselves or prosocially for another. Younger adults were self-biased, choosing to work more at higher effort levels for themselves, but also superficial, exerting less force into prosocial work. Strikingly, compared to younger adults, older people were more willing to put in effort for others and exerted equal force for self and other. Increased prosociality in older people has important implications for human behaviour and societal structure.

Mechanisms underlying visual imagery, the ability to create vivid mental representations of a scene in the absence of sensory input, remain to be fully understood. Some previous studies have proposed that visual imagery might be related to visual short-term memory (STM), with a common mechanism involving retention of visual information over short periods of time. Other observations have shown a strong relationship between visual imagery and functional activity in the hippocampus and primary visual cortex, both regions also associated with visual STM. Here we examined the relationship of visual imagery to STM and hippocampal and primary visual cortex volumes, first in a large sample of healthy people across a large age range (N = 229 behavioural data; N = 56 MRI data in older participants) and then in patients with Alzheimer's disease and Parkinson's disease (N = 19 in each group compared to 19 age-matched healthy controls). We used a variant of the "What was where?" visual object-location binding task to assess the quality of remembered information over short delays. In healthy people, no evidence of a relationship between the vividness of visual imagery and any visual STM performance parameter was found. However, there was a significant positive correlation between visual imagery and the volumes of the hippocampus and primary visual cortex. Although visual STM performance was significantly impaired in patients with Alzheimer's disease, their vividness of visual imagery scores were comparable to those of age-matched elderly controls and patients with Parkinson's disease. Despite hippocampal volumes also being reduced in Alzheimer's patients, there appeared to be no impact on their self-reported visual imagery. In conclusion, visual imagery was not significantly related to visual STM performance, either in healthy controls or Alzheimer's or Parkinson's disease but it was related to hippocampal and visual cortex volume in healthy people.

The electrohydrodynamic (EHD) force produced by ions ejected from a corona plasma is a solid state, silent mechanism for accelerating air, useful for applications ranging from electronics cooling to flying microrobots. This paper presents the theoretical motivation and the first implementation of a multi-stage, highly miniaturized EHD device, which can provide both improved absolute power output and power density as compared to single-stage devices. A laser microfabricated, folded electrode design reduces component count and assembly time. Data from one, two, and three-stage devices demonstrates a near linear scaling of output force with stage count, indicating inter-stage ducting successfully reduces losses. Device lifetime is assessed to validate the use of stainless-steel emission electrodes. Areal thrust, force density, and volumetric power density for the three-stage device are among the highest ever measured from an EHD actuator.

Human decisions are susceptible to biases, but establishing causal roles of brain areas has proved to be difficult. Here we studied decision biases in 17 people with unilateral medial prefrontal cortex damage and a rare patient with bilateral ventromedial prefrontal cortex (vmPFC) lesions. Participants learned to choose which of two options was most likely to win, and then bet money on the outcome. Thus, good performance required not only selecting the best option, but also the amount to bet. Healthy people were biased by their previous bet, as well as by the unchosen option's value. Unilateral medial prefrontal lesions reduced these biases, leading to more rational decisions. Bilateral vmPFC lesions resulted in more strategic betting, again with less bias from the previous trial, paradoxically improving performance overall. Together, the results suggest that vmPFC normally imposes contextual biases, which in healthy people may actually be suboptimal in some situations.

The goal of this review is to evaluate the current status of multi-robot systems in the context of search and rescue. This includes an investigation of their current use in the field, what major technical challenge areas currently preclude more widespread use, and which key topics will drive future development and adoption. Work blending machine learning with classical control techniques is driving progress in perception-driven autonomy, decentralized multi-robot coordination, and human–robot interaction, among others. Ad hoc mesh networking has achieved reliability suitable for safety-critical applications and may be a partial solution for communication. New modular and multimodal platforms may overcome mobility limitations without significantly increasing cost. Multi-agent systems are not currently ready for deployment in search and rescue applications; however, progress is being made in a number of critical domains. As the field matures, research should focus on realistic evaluations of constituent technologies, and on confronting the challenges of simulation-to-reality transfer, algorithmic bias in autonomous agents that rely on machine learning, and novelty-versus-reliability incentive mismatch

Reinforcement learning is a fundamental mechanism displayed by many species. However, adaptive behaviour depends not only on learning about actions and outcomes that affect ourselves, but also those that affect others. Using computational reinforcement learning models, we tested whether young (age 18–36) and older (age 60–80, total n = 152) adults learn to gain rewards for themselves, another person (prosocial), or neither individual (control). Detailed model comparison showed that a model with separate learning rates for each recipient best explained behaviour. Young adults learned faster when their actions benefitted themselves, compared to others. Compared to young adults, older adults showed reduced self-relevant learning rates but preserved prosocial learning. Moreover, levels of subclinical self-reported psychopathic traits (including lack of concern for others) were lower in older adults and the core affective-interpersonal component of this measure negatively correlated with prosocial learning. These findings suggest learning to benefit others is preserved across the lifespan with implications for reinforcement learning and theories of healthy ageing.

Nonholonomic control is a candidate to control nonlinear systems with path-dependant states. We investigate an underactuated flying micro-aerial-vehicle, the ionocraft, that requires nonholonomic control in the yaw-direction for complete attitude control. Deploying an analytical control law involves substantial engineering design and is sensitive to inaccuracy in the system model. With specific assumptions on assembly and system dynamics, we derive a Lie bracket for yaw control of the ionocraft. As a comparison to the significant engineering effort required for an analytic control law, we implement a data-driven model-based reinforcement learning yaw controller in a simulated flight task. We demonstrate that a simple model-based reinforcement learning framework can match the derived Lie bracket control - in yaw rate and chosen actions - in a few minutes of flight data, without a pre-defined dynamics function. This letter shows that learning-based approaches are useful as a tool for synthesis of nonlinear control laws previously only addressable through expert-based design.

Modular soft robots combine the strengths of two traditionally separate areas of robotics. As modular robots, they can show robustness to individual failure and reconfigurability; as soft robots, they can deform and undergo large shape changes in order to adapt to their environment, and have inherent human safety. However, for sensing and communication these robots also combine the challenges of both: they require solutions that are scalable (low cost and complexity) and efficient (low power) to enable collectives of large numbers of robots, and these solutions must also be able to interface with the high extension ratio elastic bodies of soft robots. In this work, we seek to address these challenges using acoustic signals produced by piezoelectric surface transducers that are cheap, simple, and low power, and that not only integrate with but also leverage the elastic robot skins for signal transmission. Importantly, to further increase scalability, the transducers exhibit multi-functionality made possible by a relatively flat frequency response across the audible and ultrasonic ranges. With minimal hardware, they enable directional contact-based communication, audible-range communication at a distance, and exteroceptive sensing. We demonstrate a subset of the decentralized collective behaviors that these functions make possible with multi-robot hardware implementations. The use of acoustic waves in this domain is shown to provide distinct advantages over existing solutions.

Patients with small vessel cerebrovascular disease frequently suffer from apathy, a debilitating neuropsychiatric syndrome, the underlying mechanisms of which remain to be established. Here we investigated the hypothesis that apathy is associated with disrupted decision making in effort-based decision making, and that these alterations are associated with abnormalities in the white matter network connecting brain regions that underpin such decisions. Eighty-two patients with MRI evidence of small vessel disease were assessed using a behavioural paradigm as well as diffusion weighted MRI. The decision-making task involved accepting or rejecting monetary rewards in return for performing different levels of physical effort (hand grip force). Choice data and reaction times were integrated into a drift diffusion model that framed decisions to accept or reject offers as stochastic processes approaching a decision boundary with a particular drift rate. Tract-based spatial statistics were used to assess the relationship between white matter tract integrity and apathy, while accounting for depression. Overall, patients with apathy accepted significantly fewer offers on this decision-making task. Notably, while apathetic patients were less responsive to low rewards, they were also significantly averse to investing in high effort. Significant reductions in white matter integrity were observed to be specifically related to apathy, but not to depression. These included pathways connecting brain regions previously implicated in effort-based decision making in healthy people. The drift rate to decision parameter was significantly associated with both apathy and altered white matter tracts, suggesting that both brain and behavioural changes in apathy are associated with this single parameter. On the other hand, depression was associated with an increase in the decision boundary, consistent with an increase in the amount of evidence required prior to making a decision. These findings demonstrate altered effort-based decision making for reward in apathy, and also highlight dissociable mechanisms underlying apathy and depression in small vessel disease. They provide clear potential brain and behavioural targets for future therapeutic interventions, as well as modelling parameters that can be used to measure the effects of treatment at the behavioural level.