A better understanding of how humans perceive robot personality variables could enable the design of more socially acceptable robots. In this exploratory study, we examined whether manipulations of an iCub robot’s voice and movements affected human participants’ perceptions of the robot’s personality. We programmed the robot to behave in different ways during a teaching scenario in which it played either a teaching, learning, or collaborative role, shown in recorded videos of human-robot interactions. A total of 240 participants in an Amazon Mechanical Turk study watched these videos and completed a series of questionnaires assessing their perceptions of the robot. Participants perceived the iCub as more extroverted when it spoke faster, with a higher pitch, and performed larger-amplitude movements. It was determined that participants’ personality dimensions were more influential in their perceptions of the robot’s TIPI and RoSAS personality dimensions than the robot’s social role and personality manipulations. Participants’ self-rated extroversion, emotional stability, and conscientiousness repeatedly appeared as significant factors affecting their perceptions of the robot’s personality. Interestingly, we observed strong perceiver effects, whereby participants’ perceptions of the robot’s personality traits were correlated with their own self-rated personality traits.

Human attention to dynamic visual stimuli can be measured using mouse-contingent cursor movements as a proxy for tracking human gaze when eye tracking is not feasible, for example, in online studies. FocalVid is the first such platform for collecting mouse-contingent attention data for video as a proxy approximating eye tracking. The present study analyses in detail how data collected with FocalVid are related to human gaze data. Cursor movements of 225 participants watching a variety of videos are compared to eye movements for an established corpus on human gaze that used the same videos. Such a comparison is critical to justifying the use of FocalVid or similar platforms, and for choosing suitable user interface settings to produce results most similar to human gaze. Cursor movements in FocalVid are shown to have notable similarities with eye movements. For example, distributions of cursor velocities and fixation durations are qualitatively similar to those of eye movements. However, moment-to-moment cursor movements and eye movements were significantly different with most video stimuli. We discuss the implications for mouse-contingent attention measures to video stimuli in different contexts.

Deep networks can learn to process raw sensor data and produce control output for diverse tasks. However, to leverage these models’ flexibility and expressive power, past studies have trained them on massive amounts of data. In contrast, in this work, we attempt to train deep networks from scratch with very small datasets of object pose and gripper trajectories in manipulation-task demonstrations. The same setting has previously been used in programming-by-demonstration work with specialized statistical models such as task-parameterized Gaussian mixture models (TP-GMMs). We show that deep networks can learn manipulation tasks with performance that meets or exceeds that of past statistical models, given the same small numbers of demonstrations (5-30 in our tests), without any pretraining. Data augmentation is important for good performance and training the deep networks to be equivariant to frame transformations. Transformers performed slightly better than parameter-matched long-short-term-memory (LSTM) networks, and transformers had better training and inference times. In addition to testing these methods with physical tasks, we used a family of synthetic tasks to show that larger transformer models exhibit positive transfer across dozens of tasks, performing better on each task as they are trained on others. These results suggest that deep networks are potential alternatives to TP-GMM and related methods, having the advantage of needing fewer examples per task as the number of tasks grows. The results also suggest that the large data requirements of end-to-end manipulation learning are mainly due to perceptual factors, which may help to improve the design of end-to-end systems in the future.

Gaze behaviour plays a crucial role in social interactions extending to Human-Robot Interactions. Humanoid robots equipped with anthropomorphic vision systems may be expected to employ natural gaze behaviour in interactions with humans. Here, we implemented a novel first-person human gaze state detection tool on the iCub robot and assessed its gaze detection accuracy. Using this tool, we then developed two social gaze interaction architectures for the iCub robot. The Social Gaze Space theory (SGS) of Jording and colleagues was the theoretical framework for building our first architecture, SGS Base (SGS-B). Our second architecture, SGS Interaction Architecture (SGS-IA), is the extended version of SGS Base that lets the robot take on different higher-level goals in an interaction, including being attentive to the interaction partner or attempting to direct their attention to an object. These architectures support dyadic social interactions involving initiating and responding to joint attention towards objects of interest, and are expected to allow the robot to interact in a more human-like manner in social interactions. The gaze detection accuracy of the iCub robot equipped with our choice of algorithms exceeded that of previously implemented methods. System validation trials confirmed that the robot detects the social gaze states correctly in the majority of the instances, allowing for behavioural control according to the Social Gaze Space theory. Validation tests presented in this article demonstrate the functionality of the SGS Interaction Architecture and also highlight the differences in personal interaction dynamics when the humanoid robot has no particular goals as opposed to when it has interactional goals appropriate to a teaching scenario.

Traditional medical artificial intelligence models that are approved for clinical use restrict themselves to single-modal data (e.g., images only), limiting their applicability in the complex, multimodal environment of medical diagnosis and treatment. Multimodal transformer models in health care can effectively process and interpret diverse data forms, such as text, images, and structured data. They have demonstrated impressive performance on standard benchmarks, like United States Medical Licensing Examination question banks, and continue to improve with scale. However, the adoption of these advanced artificial intelligence models is not without challenges. While multimodal deep learning models like transformers offer promising advancements in health care, their integration requires careful consideration of the accompanying ethical and environmental challenges.

Imitation learning methods have proven effective in learning robotic tasks by leveraging multiple human-controlled demonstrations. However, existing approaches often struggle to generalize across a wide range of tasks, such as extrapolating to unseen object locations, incorporating via-point modulation, accurately modeling orientation, handling trajectories with multiple options, and capturing aiming actions. In this study, we propose a novel framework that combines ideas from task-parameterized Gaussian mixture models and probabilistic movement primitives to address these limitations and satisfy all the aforementioned properties within a single framework. We conduct comprehensive evaluations of our approach on four real-life tasks: pick-and-place, water pouring, shooting a hockey puck into a net, and sweeping.

The diverse functions of different cortical areas are thought to arise from their distinct groups of inputs. However, additional organizing principles may exist in the spatial structure of converging inputs. We investigated spatial convergence patterns of projections from primary sensory areas to other areas throughout the mouse isocortex. We used a large tract tracing dataset to estimate the dimension of the space into which topographical connections from multiple modalities converged within each other cortical area. We call this measure the topography dimension (TD). TD is higher for areas that receive inputs of similar strength from multiple sensory modalities, and lower when multiple inputs terminate in register with one another. Across the isocortex, TD varied by a factor of ~4. TD was positively correlated with hierarchy score, an independent measure that is based on laminar connection patterns. Furthermore, TD (an anatomical measure) was significantly related to several measures of neural activity. In particular, higher TD was associated with higher neural activity dimension, lower population sparseness, and lower lifetime sparseness of spontaneous activity, independent of an area's hierarchical position. Finally, we analyzed factors that limited TD and found that linear correlations among projections from different areas typically had little impact, while diversity of connection strengths, both between different projections onto the same area, and within projections across different parts of an area, limited TD substantially. This analysis revealed additional intricacy of cortical networks, beyond areas' sets of connections and hierarchical organization. We propose a means of approximating this organization in deep-network models. ### Competing Interest Statement The authors have declared no competing interest.

The ability to encode meaningful structure into deep learning models opens up the potential for incorporating prior knowledge, particularly in fields where domain-specific information is of great importance. However, transformer models trained on NLP tasks with medical data often have randomly initialized embeddings that are then adjusted based on training data. For terms appearing infrequently in the dataset, there is less opportunity to improve these representations and learn semantic similarity with other concepts. Medical ontologies already represent many of the biomedical concepts and define a relationship structure between these concepts, making ontologies a valuable source of domain-specific information. One of the ongoing challenges of deep learning is finding methods to incorporate this domain knowledge into models. Holographic Reduced Representations (HRR) are capable of encoding ontological structure by composing atomic vectors to create structured higher-level concept vectors. Deep learning models can further process these structured vectors without needing to learn the ontology from training data. We developed an embedding layer that generates concept vectors for clinical diagnostic codes by applying HRR operations that compose atomic vectors based on the SNOMED CT ontology. This approach still allows for learning to update the atomic vectors while maintaining structure in the concept vectors. We trained a Bidirectional Encoder Representations from Transformers (BERT) transformer model to process sequences of clinical diagnostic codes and used the resulting HRR concept vectors as the embedding matrix for the model. The model was first pre-trained on a masked-language modeling (MLM) task before being fine-tuned for mortality and disease prediction tasks. The HRR-based approach improved performance on the pre-training and fine tuning tasks compared to standard transformer embeddings. This is the first time HRRs have been used to produce structured embeddings for transformer models and we find that this approach maintains semantic similarity between medically related concept vectors and allows better representations to be learned for rare codes in the dataset, as rare codes are composed of elements that are shared with more common codes.

Deep convolutional neural networks are important models of the visual cortex that account relatively well for brain activity and are able to perform ethologically relevant functions. However, it is unknown which combination of factors, such as network architecture, training objectives, and data best align this family of models with the brain. Here we investigate the statistics of training data. We hypothesized that stimuli that are naturalistic for mice would lead to higher similarity between deep network models and activity in mouse visual cortex. We used a video-game engine to create training datasets in which we varied the naturalism of the environment, the movement statistics, and the optics of the modelled eye. The naturalistic environment substantially and consistently led to greater brain similarity, while the other factors had more subtle and area-specific effects. We then hypothesized that differences in brain similarity between the two environments arose due to differences in spatial frequency spectra, distributions of color and orientation, and/or temporal autocorrelations. To test this, we created abstract environments, composed of cubes and spheres, that resembled the naturalistic and non-naturalistic environments in these respects. Contrary to our expectations, these factors accounted poorly for differences in brain similarity due to the naturalistic and non-naturalistic environments. This suggests that the higher brain similarities we observed after training with the naturalistic environment were due to more complex factors. ### Competing Interest Statement The authors have declared no competing interest.

Under difficult viewing conditions, the brain’s visual system uses a variety of recurrent modulatory mechanisms to augment feedforward processing. One resulting phenomenon is contour integration, which occurs in the primary visual (V1) cortex and strengthens neural responses to edges if they belong to a larger smooth contour. Computational models have contributed to an understanding of the circuit mechanisms of contour integration, but less is known about its role in visual perception. To address this gap, we embedded a biologically grounded model of contour integration in a task-driven artificial neural network and trained it using a gradient-descent variant. We used this model to explore how brain-like contour integration may be optimized for high-level visual objectives as well as its potential roles in perception. When the model was trained to detect contours in a background of random edges, a task commonly used to examine contour integration in the brain, it closely mirrored the brain in terms of behavior, neural responses, and lateral connection patterns. When trained on natural images, the model enhanced weaker contours and distinguished whether two points lay on the same versus different contours. The model learned robust features that generalized well to out-of-training-distribution stimuli. Surprisingly, and in contrast with the synthetic task, a parameter-matched control network without recurrence performed the same as or better than the model on the natural-image tasks. Thus, a contour integration mechanism is not essential to perform these more naturalistic contour-related tasks. Finally, the best performance in all tasks was achieved by a modified contour integration model that did not distinguish between excitatory and inhibitory neurons.

Abstract Background and Aims Hypertension is the number one risk factor for premature death worldwide. Artificial Intelligence (AI) Clinical Decision Support Systems are an important next step for hypertension management but require rigorous evaluation before assimilation into routine clinical practice. Our aim is to develop and evaluate an AI clinical decision support tool for hypertension management trained on randomised clinical trial data. Method The Systolic Blood Pressure Intervention Trial (SPRINT) trial showed that intensive BP control to SBP <120 mm Hg results in significant cardiovascular benefit in high-risk patients with hypertension compared with routine BP control to <140 mm Hg. We trained a feed-forward neural network using Keras and Tensorflow for R and data from 9361 persons in the SPRINT randomized clinical trial to predict the probability of increase, reduce or no-change in total number of anti-hypertensive medications at each visit. The network is designed to model SPRINT investigator deviations from the protocol. Six baseline patient variables (age, sex, race, aspirin use, eGFR, group assignment (intensive or standard)), two visit patient variables (Systolic and Diastolic Blood Pressure), and two previous visit variables (Systolic and Diastolic Blood Pressure) were inputted into the model after data normalisation and centering. Hyperparameters were tuned using a grid search method. We conducted internal validation using a 20% validation set. Clinical validation was performed using an unseen test set of typical hypertension scenarios (n = 50). An R Shiny app was developed to enter new patient information and display a sensitivity analysis of probabilities calculated after changing each baseline variable by plus and minus 10 (Figure 1). To avoid poor performance on new out of distribution cases we tested five methods for Out of Distribution (OOD) detector, 1. Autoencoders, 2. Normalizing flow, 3. Local Outlier Factor (LOF), 4. Probabilistic Principal Component Analysis (PPCA) and Kernel Density Estimation (KDE). Results The accuracy was 76.7% on the validation dataset after 1000 Epochs. Clinical validation revealed suboptimal performance for unseen Out of Distribution data (e.g. recommending reduction of medication when SBP >160 and number of medications was zero). This poor performance on OOD samples prompted the creation of an OOD detector to use in series with the AI Decision Support System for Hypertension. The AUROCs were 0.77 for AE, 0.96 for Normalizing Flow, 0.82 for LOF, 0.88 for PPCA and 0.96 for the Kernel Density Estimation. Conclusion Artificial Intelligence (AI) Clinical Decision Support Systems for Hypertension are feasible and an important next step for hypertension management but generalisability and safety require rigorous evaluation before assimilation into routine clinical practice.

This abstract explores classroom Human-Robot Interaction (HRI) scenarios with an emphasis on the adaptation of human-inspired social gaze models in robot cognitive architecture to facilitate a more seamless social interaction. First, we detail the HRI scenarios explored by us in our studies followed by a description of the social gaze model utilized for our research. We highlight the advantages of utilizing such an attentional model in classroom HRI scenarios. We also detail the intended goals of our upcoming study involving this social gaze model.

Convolutional neural networks trained on object recognition derive inspiration from the neural architecture of the visual system in mammals, and have been used as models of the feedforward computation performed in the primate ventral stream. In contrast to the deep hierarchical organization of primates, the visual system of the mouse has a shallower arrangement. Since mice and primates are both capable of visually guided behavior, this raises questions about the role of architecture in neural computation. In this work, we introduce a novel framework for building a biologically constrained convolutional neural network model of the mouse visual cortex. The architecture and structural parameters of the network are derived from experimental measurements, specifically the 100-micrometer resolution interareal connectome, the estimates of numbers of neurons in each area and cortical layer, and the statistics of connections between cortical layers. This network is constructed to support detailed task-optimized models of mouse visual cortex, with neural populations that can be compared to specific corresponding populations in the mouse brain. Using a well-studied image classification task as our working example, we demonstrate the computational capability of this mouse-sized network. Given its relatively small size, MouseNet achieves roughly 2/3rds the performance level on ImageNet as VGG16. In combination with the large scale Allen Brain Observatory Visual Coding dataset, we use representational similarity analysis to quantify the extent to which MouseNet recapitulates the neural representation in mouse visual cortex. Importantly, we provide evidence that optimizing for task performance does not improve similarity to the corresponding biological system beyond a certain point. We demonstrate that the distributions of some physiological quantities are closer to the observed distributions in the mouse brain after task training. We encourage the use of the MouseNet architecture by making the code freely available.

Spiking Neural Networks (SNNs) are an emerging domain of biologically inspired neural networks that have shown promise for low-power AI. A number of methods exist for building deep SNNs, with Artificial Neural Network (ANN)-to-SNN conversion being highly successful. MaxPooling layers in Convolutional Neural Networks (CNNs) are an integral component to downsample the intermediate feature maps and introduce translational invariance, but the absence of their hardwarefriendly spiking equivalents limits such CNNs' conversion to deep SNNs. In this paper, we present two hardware-friendly methods to implement Max-Pooling in deep SNNs, thus facilitating easy conversion of CNNs with MaxPooling layers to SNNs. In a first, we also execute SNNs with spiking-MaxPooling layers on Intel's Loihi neuromorphic hardware (with MNIST, FMNIST, & CIFAR10 dataset); thus, showing the feasibility of our approach.

Driving Scene Understanding is a broad field which addresses the problem of recognizing a variety of on-road situations; namely driver behaviour/intention recognition, driver-action causal reasoning, pedestrians’ and nearby vehicles’ intention recognition, etc. Many existing works propose excellent AI based solutions to these interesting problems by leveraging visual data along with other modalities. However, very few researchers venture into determining the necessary metadata of the visual inputs to their models. This work attempts to put forward some useful insights about the required spatial resolution and temporal context/depth of the visual data for Driving Scene Understanding.

Foveation is an important part of human vision, and a number of deep networks have also used foveation. However, there have been few systematic comparisons between foveating and non-foveating deep networks, and between different variable-resolution downsampling methods. Here we define several such methods, and compare their performance on ImageNet recognition with a custom Densenet network. The best variable-resolution method slightly outperforms uniform downsampling. Thus in our experiments, foveation does not substantially help or hinder object recognition in deep networks.

Background Freezing of gait (FOG) is arguably the most disabling motor symptom experienced with Parkinson's disease (PD), but treatments are extremely limited due to our poor understanding of the underlying mechanisms. Three cortical domains are postulated in recent research (ie, the cognitive, limbic, and sensorimotor domains), thus, treatments targeting these mechanisms of FOG may potentially be effective. Cognitive training, cognitive behavioral therapy (CBT, a well-known anxiety intervention), and proprioceptive training may address the cognitive, limbic, and sensorimotor domains, respectively. Objective To investigate whether these 3 treatments could improve functional outcomes of FOG. Methods In a single-blind, randomized crossover design, 15 individuals with PD and FOG were randomized into different, counterbalanced orders of receiving the interventions. Each consisted of eight 1-hour sessions, twice weekly for 4 weeks. FOG severity was assessed as the primary outcome using a novel gait paradigm that was aimed at evoking FOG when the cognitive, limbic, or sensorimotor domains were independently challenged. Results FOG severity significantly improved after the cognitive intervention, with strong trends toward improvement specifically in the baseline and cognitive-challenge assessment conditions. CBT, as the anxiety intervention, resulted in significantly worse FOG severity. In contrast, proprioceptive training significantly improved FOG severity, with consistent trends across all conditions. Conclusions The cognitive and proprioceptive treatments appeared to improve different aspects of FOG. Thus, either of these interventions could potentially be a viable treatment for FOG. However, although the results were statistically significant, they could be sensitive to the relatively small number of participants in the study. Considering the significant results together with nonsignificant trends in both FOG and gait measures, and given equal time for each intervention, proprioceptive training produced the most consistent indications of benefits in this study. (clinicaltrials.gov NCT03065127).