Touchless interactions, like mid-air gestures, show promise in enhancing interaction and communication for individuals with physical disabilities. This study explores how mid-air gestural interaction can improve the accessibility of touchscreen devices, like tablets and other touchscreen tablet-like devices, for individuals with Multiple Sclerosis and dexterity impairments (MS-Dex). Two studies were conducted. The first, assessed the challenges faced by ten MS-Dex participants with conventional tablet interactions. The second, adopting a user-centered design approach, explored mid-air gesture preferences among ten MS-Dex participants, aiming to develop alternative, more accessible interaction methods and to identify potential gestures for a touchless vocabulary. Gesture elicitation sessions were conducted, during which video recordings and Leap Motion Controller tracking were employed to record gesture metrics. This data was used to identify natural hand gestures suitable for 18 common tablet interaction scenarios. Both studies were compared holistically to understand the interplay between user challenges and gesture preferences, informing the development of effective and accessible touchless interaction solutions. The first study identified significant difficulties with activities requiring extensive movement and coordination. The second study showed that the elicited gestures were perceived as effective, intuitive, and easy to execute, confirming their accessibility for MS-Dex participants. Preference for gestures that minimized precise finger coordination and extensive hand movements was noted. This research contributes to understanding the physical aspects of mid-air gestural interactions, suggesting design directions for more accessible tablet interfaces and providing insights into developing a standardized touchless gesture vocabulary, ultimately promoting greater accessibility for individuals with MS-Dex.

Amidst the mounting challenges posed by the ageing of worldwide populations, innovative care approaches need to be explored. Socially Assistive Robots (SARs) have emerged as a potential solution, however despite their potential benefits, they have yet to realise widespread utilisation. We conducted a mixed-methodology study to explore the perspectives of 24 support workers and other healthcare professionals on SARs and the potential integration of “needy” traits to increase acceptance of future SARs. Needy traits in this context refers to the integration of traits which produce a sense of dependence from the robot to the user, with the intention of fostering similar engagements as between a person and their pet. Focus groups, questionnaires and interviews were utilised to gather valuable insights. Overall, support workers exhibited a positive attitude towards existing SARs and were receptive to the concept of an SAR with “needy” traits. However, several significant barriers including privacy, technology literacy, and cost still need to be addressed to allow for their widespread adoption.

As the average age of developed countries continues to increase the methods used to care for these frail individuals needs to adapt to keep up with demand. Socially Assistive Robots (SAR) are a potential solution to this problem, however they have been held back by adoption issues for a number of reasons. One of the biggest barriers to adoption is the lack of acceptance by their intended users. Through this literature review we intend to explore the factors that influence SAR acceptance and the work that has been done into the implementation of a robot with “needy” characteristics. Research papers were gathered from some of the leading research databases for computer science that were published between 2004 and 2022, as that was the initial launch of Paro, one of the first SARs to see widespread success. We conducted a search for papers on the topic of SAR acceptance for the elderly or otherwise vulnerable and found a total of 52 that matched our acceptance criteria. This review highlighted some of the ways that SARs can be adapted towards specific demographics in order to increase acceptance. We have also identified the current gap in the research for studies focused around the concept of a “needy” robot. While aspects of that kind of design have been viewed positively in several of the studies found here, we found none that focus on this concept as a primary research aim, indicating that there is value in further investigation of this type of SAR.

Accessibility research, and the experience of disabled people are increasingly threatened by economic cuts and fluctuating political landscapes. Austerity sees a shift towards frugality with public money, with disabled people most at risk. Accessibility research, similarly, faces challenges as governments wish to reduce research funding, and large technology companies shift their focus to the global AI race. This workshop will bring together accessibility researchers, students, and practitioners to explore what we can do with these increasing constraints in the UK and its broader context. We will create a strategy for both managing current challenges and resisting future constraints, with a view to shaping a community of resilient and impact-led accessibility researchers in the UK.

We present a tried and tested methodology for simulating the behaviour of robots within the built environment. 3D scans are taken, converted into CAD like models and used within an emulated environment with real robot models. The simulations show movement behaviours, sight capabilities and uncover limitations that robots will undergo within specific environments. Using this method we are able to produce a largely scalable, time efficient way of understanding both robot ability with a built environment and indications of how a co-habitation with a human resident will take place. We produce some limitations of the methodology as well as its efficacy to replace a manual method of installing robots within real physical environments (something that is not only time consuming but also cost or facility prohibitive).

For individuals with Multiple Sclerosis (MS) and dexterity impairments (MS-Dex), touchscreen interaction can be particularly challenging, as symptoms like finger numbness and impaired motion regulation negatively impact smartphone usability. This study investigated the usability and accessibility of a mid-air gesture interaction technique for MS-Dex. Touchscreen technologies are ubiquitous, but these individuals often face challenges with touch-based interaction. Touchless input methods, like mid-air gestures, offer a promising alternative for enhancing interaction and communication with touch-based technology. We developed an interactive tool (HF-MAI) using hand and finger mid-air gestures for common touchscreen tasks, using the Leap Motion Controller (LMC) for gesture recognition. MS-Dex individuals, MS therapists, and a usability and user experience (UX) expert participated in the evaluation. We assessed usability, user satisfaction, and accessibility of the HF-MAI system. Results indicate HF-MAI is highly usable and accessible; participants performed gestures successfully and reported high satisfaction. Notably, pinch, spread, and rotate (two-finger/single-finger) mid-air gestures were performed with faster completion times, contrasting with the difficulty of such gestures on touchscreens. The gesture vocabulary, particularly the adaptation to single/two-finger movements, facilitated easier interaction, reducing difficulty for users with limited dexterity. This study demonstrates the potential of mid-air gestures using co-created vocabularies to enhance accessibility for individuals with MS, offering a promising alternative to traditional touch-based input. This technology can significantly improve their access to touchscreens, enhancing independence and quality of life.

Inclusive user experience design that caters to a broad range of users can enhance the ease of interaction with systems. Despite progress, certain systems like smart voice assistants often remain inaccessible to some user types, particularly individuals with dysarthria whose speech may not be recognized by these devices. In this article, we present a novel interaction technique that relies on non-verbal voice cues instead of words and sentences. To evaluate the usability, user experience, and workload of this approach, we developed a system (DARIA) and conducted two studies, exploring two different design options. The first study tested the system without a customization option, while the second introduced a customization option. We used three questionnaires and conducted post-study interviews for thorough evaluation. The findings indicated user acceptance of this interaction technique in general was high. When compared with a market leader product (Alexa), DARIA showed higher performance in terms of accessibility. Users found the non-verbal system to be more usable and likable. Interestingly, the customizable design was found less usable than the pre-defined.

The integration of digital serious games into speech learning (rehabilitation) has demonstrated significant potential in enhancing accessibility and inclusivity for children with speech disabilities. This review of the state of the art examines the role of serious games, Artificial Intelligence (AI), and Natural Language Processing (NLP) in speech rehabilitation, with a particular focus on interaction modalities, engagement autonomy, and motivation. We have reviewed 45 selected studies. Our key findings show how intelligent tutoring systems, adaptive voice-based interfaces, and gamified speech interventions can empower children to engage in self-directed speech learning, reducing dependence on therapists and caregivers. The diversity of interaction modalities, including speech recognition, phoneme-based exercises, and multimodal feedback, demonstrates how AI and Assistive Technology (AT) can personalise learning experiences to accommodate diverse needs. Furthermore, the incorporation of gamification strategies, such as reward systems and adaptive difficulty levels, has been shown to enhance children’s motivation and long-term participation in speech rehabilitation. The gaps identified show that despite advancements, challenges remain in achieving universal accessibility, particularly regarding speech recognition accuracy, multilingual support, and accessibility for users with multiple disabilities. This review advocates for interdisciplinary collaboration across educational technology, special education, cognitive science, and human–computer interaction (HCI). Our work contributes to the ongoing discourse on lifelong inclusive education, reinforcing the potential of AI-driven serious games as transformative tools for bridging learning gaps and promoting speech rehabilitation beyond clinical environments.

Rehabilitation and exercises for children with speech disabilities can be tedious. To address this issue, we created two different types of video games that will hopefully act as a motivation to children to keep playing while also practicing their words. The games were created using a user-centered approach with experts on accessibility and speech rehabilitation. We used pre-trained natural language models to detect the word the children are saying. The performance of their game is correlated to the performance of their articulation of the words; namely, the confidence level of the spoken word to the representation of the word in our pretrained model. We are now in the process of testing the effects of these games in the rehabilitation process of children with speech disabilities.

Software accessibility has recently become a legal requirement in most countries. Minimal guidance into the legalities of this matter has been provided, leaving most software engineers to adapt software they write to “skim” the surface of what is acceptable, such as providing simple screen reading capability. Even less information is available for users with two disabilities. In this work, we provide a user-centered designed game example, with a participatory design approach, to address not one but two disabilities, vision and dexterity impairments, concurrently. We show how entertainment and interaction can take priority in the development process while not compromising on the usability and accessibility of the games. We provide the reader with seven guidelines from our findings to create accessible mobile games that can be entertaining for people with dual disabilities and non-disabled individuals equally.

Emotions play an important role in human-computer interaction, but there is limited research on affective and emotional virtual agent design in the area of teaching simulations for healthcare provision. The purpose of this work is twofold: firstly, to describe the process for designing affective intelligent agents that are engaged in automated communications such as person to computer conversations, and secondly to test a bespoke prototype digital intervention which implements such agents. The presented study tests two distinct virtual learning environments, one of which was enhanced with affective virtual patients, with nine 3rd year nursing students specialising in mental health, during their professional practice stage. All (100%) of the participants reported that, when using the enhanced scenario, they experienced a more realistic representation of carer/patient interaction; better recognition of the patients' feelings; recognition and assessment of emotions; a better realisation of how feelings can affect patients' emotional state and how they could better empathise with the patients.

People living with disabilities of dexterity can be vulnerable to attackers when authenticating using physical input methods, such as when inputting PIN numbers using a keypad at an ATM(Cash Point), due to the extended time these interactions take because of the device’s lack of accommodations and accessibility. This makes their input more observable to a potential attacker and thus compromises their security. In addition, when ease of use is severely compromised, this may cause a need to circumvent good security practices for practical usability which further makes these individuals vulnerable to potential attackers. While research in the field of accessible and secure authentication exists, limited work has focused on the unique needs of individuals who have limited to no hand or finger dexterity. This paper proposes an accessible framework for authentication (AAFIDD), that focuses on meeting the needs of this group. We implemented a prototype authentication model and present an initial user study with 7 participants that evaluated the efficacy of this prototype and the framework. Each participant was randomly assigned a PIN and asked to input it using a method reliant on hand-dexterity and then using the prototype gaze-based input. Users were timed and asked to evaluate their experience in terms of ease of use while a researcher attempted to perform an over-the-shoulder attack to evaluate the security. We found that the prototype input method was less likely to be interpreted by an observer than using a mouse to input, while users considered the prototype input method accessible and easy to use.

This study explores the effectiveness and user experience of different interaction methods used by individuals with dysarthria when engaging with Smart Virtual Assistants (SVAs). It focuses on three primary modalities: direct speech commands through Alexa, non-verbal voice cues via the Daria system, and eye gaze control. The objective is to assess the usability, workload, and user preferences associated with each method, catering to the varying communication capabilities of individuals with dysarthria. While Alexa and Daria facilitate voice-based interactions, eye gaze control offers an alternative for those unable to use voice commands, including users with severe dysarthria. This comparative approach aims to determine how the usability of each interaction method varies, conducted with eight participants with dysarthria. The results indicated that non-verbal voice interactions, particularly with the Daria system, were favored because of their lower workload and ease of use. The eye gaze technology, while viable, presented challenges in terms of the higher workload and usability. These findings highlight the necessity of diversifying interaction methods with SVAs to accommodate the unique needs of individuals with dysarthria.

The design of age-friendly homes benefits vulnerable groups, such as frail people and older adults. Advances in smart home technologies, including robots, have important synergies with homes designed for health needs. Yet, focus on environmental and sustainable housing design and improvements misses important opportunities for collective impact. Stronger involvement of disciplines, such as those from the built environment for technological integration within homes and effects on space and the community, is needed. There is a need for a unified framework integrating the needs and factors of the resident, smart home technologies and robots, and the built environment, and that includes the concept of a “home”. With the remodeling of housing towards sustainable and environmental targets, as well as advances in smart home technologies such as robots, the timeliness of shared input for the benefit of residents now and in the future is of the essence. This would help target future research into effective and optimized cohabitation with technology within homes for the purpose of improving the wellbeing of residents.

Abstract Visually-induced dizziness (visual vertigo) is a core symptom of Persistent Perceptual Postural Dizziness (PPPD) and occurs in other conditions and general populations. It is difficult to treat and lacks new treatments and research. We incorporated the existing rehabilitation approach of visual desensitisation into an online game environment to enhance control over visual motion and complexity. We report a mixed-methods feasibility trial assessing: Usage and adherence; rehabilitation potential; system usability and enjoyment; relationship with daily dizziness. Participants played online with (intervention, N = 37) or without (control, N = 39) the visual desensitisation component for up to 5–10 min, twice daily for 6 weeks. Dropout was 45%. In the intervention group, N = 17 played for the recommended time while N = 20 played less. Decreases in visual vertigo symptoms, anxiety and depression correlated with playtime for the intervention but not control. System usability was high. Daily symptoms predicted playtime. Qualitative responses broadly supported the gamified approach. The data suggest gamified visual desensitisation is accessible, acceptable and, if adherence challenges can be overcome, could become a useful addition to rehabilitation schedules for visually-induced dizziness and associated anxiety. Further trials are needed.

In recent years, rapid advancements have taken place for automatic speech recognition (ASR) systems and devices. Though ASR technologies have increased, the accessibility of these novel interaction systems is underreported and may present difficulties for people with speech impediments. In this article, we attempt to identify gaps in current research on the interaction between people with dysarthria and ASR systems and devices. We cover the period from 2011, when Siri (the first and the leading commercial voice assistant) was launched, to 2020. The review employs an interaction framework in which each element (user, input, system, and output) contributes to the interaction process. To select the articles for review, we conducted a search of scientific databases and academic journals. A total of 36 studies met the inclusion criteria, which included use of the word error rate (WER) as a measurement for evaluating ASR systems. This review determines that challenges in interacting with ASR systems persist even in light of the most recent commercial technologies. Further, understanding of the entire interaction process remains limited; thus, to improve this interaction, the recent progress of ASR systems must be elucidated.

Our work provides a novel method for rich information discovery about the evolution of technical fields and company developments through patent relationships. A new exploratory method and graphical tool to discover technology proximity based on patent classification information are introduced. By technology we mean a technical field (defined by an International Patent Classification-IPC-code or a combination of them) or an organization (such as a tech company, research center, or institution). A single data structure is used for characterizing both technical fields and organizations, to visualize them as items of the very same body. This new method generates two graphs: the first graph, the TechnologyMap, visualizes technology items in a 2D plot wherein technical fields and companies will appear positioned relative to each other; the. A second graph, the Focused TechnologyMap, visualizes technology items with respect to a selected one, the focus, which is located in the center of a circle whose radii correspond to the complete set of IPC codes. This article represents the process and algorithms used for production of the graphs, and solidifies the assumptions of validity by presenting two of the many successful test cases to which it was applied.

People with speech impairments usually use assistive technology devices to assist them with communication and daily tasks. These devices can be controlled using different modalities, such as touch, eye gaze, gestures, and others. This article proposes a standardized methodology for designing non-verbal voice cue interactive systems that enable people with dysarthria to vocally interact with virtual home assistants (VHAs). We adopted a qualitative data-gathering approach to gain insights into users’ experiences and requirements and to determine crucial design elements for designing interactive voice assistants for people with dysarthria. Nineteen participants with varying levels of dysarthria took part in the study to create the framework. A system was then built using the proposed framework, and an additional test was performed with a further seven participants to validate the created system, thus inferring the validity of the framework. Our work empirically demonstrates how an informed, structured design of a fast, direct (verbal rather than forcing users to change modalities or using an intermediate device) method of communication improves the usability of VHAs for people with dysarthria while simultaneously allowing for a more authentic experience. The current data also highlighted that using non-verbal voice cues would be a convenient option. By providing a reproducible framework for developing non-verbal interactive systems for VHAs, we can increase the accessibility of said devices, their usability, and their user experience.