A significant portion of the global population lives with Severe Speech and Motor Impairment (SSMI), which profoundly restricts their ability to interact with the physical and social environment. Such limitations often lead to social isolation and a loss of agency. In alignment with the World Health Organization (WHO) framework, assistive technology should aim to improve individual functioning and enable social inclusion. This research presents a Socially Assistive Robotic (SAR) framework designed to bridge this gap through a gamified, eye-gaze controlled Human-Robot Interaction (HRI) system. The proposed system utilizes a low-cost eye-tracker to allow users to manipulate robotic agents—specifically a four-way RC car and a 4-Degree of Freedom (DoF) robotic arm—within a Mixed Reality (MR) and Virtual Reality (VR) environment. These robotic platforms serve as social proxies, enabling users to regain a sense of embodied agency in shared spaces. Several individuals with SSMI (including conditions such as Cerebral Palsy and Spinal Cord Injury) participated in a series of user trials. The study evaluated the system’s impact on user engagement, perceived social presence, and functional performance through statistical significance testing of task completion times and qualitative assessments. Feedback was gathered in a qualitative manner along with quantitative performance metrices. Results indicate that the gamified SAR approach significantly reduces the perceived “uncanny valley” effect often associated with assistive interfaces and fosters a high degree of user motivation. This work contributes to the field of social robotics by demonstrating how mediated robotic embodiment can restore communicative and social functioning for the most vulnerable populations, providing a scalable model for inclusive HRI design.
Social robots are designed to assist people and enhance their quality of life. To fulfil this task, they must earn users’ trust and acceptance by providing an engaging experience and an assistive function. Therefore, designing social robots with natural and human-like behaviour might facilitate intuitive natural human-robot interaction, fostering better understanding and human-robot bonding. However, achieving natural interaction requires considering many biological and social aspects that increase the robot’s behavioural complexity when operating in dynamic environments. This paper presents a biologically inspired Deep Reinforcement Learning system for autonomous robots acting in complex social environments. The system includes a biologically inspired model that emulates some human biological functions, including physiological, psychological, affective, and social processes whose dynamics are affected by the robot’s actions and perceived stimuli. The robot learns to map the different states of the robot to 14 actions to maintain optimal well-being. We compare eight state-of-the-art Deep Reinforcement Learning simulation methods to find the best results in our environment. The results of this study show that Rainbow’s algorithm provides the best learning speed and stability results. We then integrated the Rainbow model into the Mini social robot to assess whether its behaviour supports well-being optimisation. Finally, we conducted a preliminary user evaluation with 15 participants using the Artificial-Social-Agent questionnaire to analyse how users perceive Mini’s behaviour using the Deep Reinforcement Learning model. Results suggest overall positive user ratings, highlighting the dimensions related to the robot’s naturalness, behaviour generation, and social attributes.
Humans and robots are increasingly coming into contact in unexpected ways as advances in robotics move autonomous machines from the lab to daily living spaces. Despite a wide range of self-report measures being used in human-robot interactions (HRI) research, many instruments are not fully validated or may be only appropriate for a specialized type of robot or interaction. The goal of this paper is to provide a systematic review of validated HRI self-report measures from 2000 to 2025 (n = 96) along with a classification of the purpose of each measure to facilitate researchers evaluating which existing measure might best suit their own study design. In addition, we specifically investigate the selected measures for characteristics that would be appropriate to use in human-robot encounters, which are distinct from human-robot interactions. Based on the systematic review results, we contribute an open-source database of the self-report measures and a set of recommendations for researchers who may be deciding which measures might best fit their study goals. The repository can be accessed at: ( https://web.colby.edu/insitelab/hri-repo/ )
To recognize continuous hand gestures from real-time video streams quickly and enable natural human-robot interaction, this paper proposes continuous hand gesture-based human-robot interaction for indoor mobile robot based on Multi-scale Local and Global skeleton Spatiotemporal feature extraction Network (MLGSNet). Firstly, we propose spatiotemporal feature extraction method of hand skeletal joint flow based on uniform sampling to address the challenge of extracting spatiotemporal features from videos. Subsequently, we propose multi-scale local and global spatiotemporal graph convolution network for hand gesture recognition to effectively capture both local and global dynamic spatiotemporal features during gesture execution, which is characterized by combining multi-scale attention, graph and temporal convolution. Furthermore, we design hand gesture activation based on mean filtering of dual confidence to accurately activate continuous gesture streams and recognize gesture command units under uneven gesture stream distribution. Experimental results on two public datasets demonstrate that the MLGSNet achieves state-of-the-art performance in both isolated and continuous gesture recognition. Finally, case study on the robot platform shows an overall human-robot interaction performance of 93.05
Socially aware robots must adhere to human social norms, such as respecting social groups by avoiding robot intrusions. Consequently, identifying individuals and social groups constitutes a critical perception task for social navigation. However, existing laser scanner-based perception methods face challenges such as partial occlusion and high latency. In this study, an improved laser scanner-based neural radiance field (NeRF) is proposed to identify social groups with two-dimensional (2D) laser scanner inputs. This NeRF method utilizes a distance field to reconstruct partially occluded laser points and a density field to filter out distant noise. To further minimize identification errors in dynamic and crowded environments, a novel social interaction field is introduced, which assesses social features including motion direction consistency and density invariance. The improved NeRF method has been validated in both public datasets and real-world experiments. Experimental results show that the proposed method achieves equal error rates (EERs) of 0.213, 0.170, and 0.195 in laboratory, corridor, and outdoor environments, respectively. Furthermore, the method achieves occlusion recovery rates (ORR) of 84.2
