Designing Conversational AI systems to support older adults requires these systems to explain their behavior in ways that align with older adults' preferences and context. While prior work has emphasized the importance of AI explainability in building user trust, relatively little is known about older adults' requirements and perceptions of AI-generated explanations. To address this gap, we conducted an exploratory Speed Dating study with 23 older adults to understand their responses to contextually grounded AI explanations. Our findings reveal the highly context-dependent nature of explanations, shaped by conversational cues such as the content, tone, and framing of explanation. We also found that explanations are often interpreted as interactive, multi-turn conversational exchanges with the AI, and can be helpful in calibrating urgency, guiding actionability, and providing insights into older adults' daily lives for their family members. We conclude by discussing implications for designing context-sensitive and personalized explanations in Conversational AI systems.

Designing Conversational AI systems to support older adults requires more than usability and reliability, it also necessitates robustness in handling conversational breakdowns. In this study, we investigate how older adults navigate and repair such breakdowns while interacting with a voice-based AI system deployed in their homes for medication management. Through a 20-week in-home deployment with 7 older adult participant dyads, we analyzed 844 recoded interactions to identify conversational breakdowns and user-initiated repair strategies. Through findings gleaned from post-deployment interviews, we reflect on the nature of these breakdowns and older adults' experiences of mitigating them. We identify four types of conversational breakdowns and demonstrate how older adults draw on their situated knowledge and environment to make sense of and recover from these disruptions, highlighting the cognitive effort required in doing so. Our findings emphasize the collaborative nature of interactions in human-AI contexts, and point to the need for AI systems to better align with users' expectations for memory, their routines, and external resources in their environment. We conclude by discussing opportunities for AI systems to integrate contextual knowledge from older adults' sociotechnical environment and to facilitate more meaningful and user-centered interactions.

The caregiving environment for an older adult aging in place includes a network of caregivers working with the older adult to support their needs and maintain independence. As older adults experience cognitive and functional changes, their caregiving network expands to include spouses or siblings (who are often older adults themselves), children, friends, neighbors and community members-each bringing unique values, expectations, and goals. In this network of care, technology-enabled support offers the potential to mediate care responsibilities, such as coordinating activities and assisting with everyday tasks. However, designing these systems requires addressing value tensions among caregivers, cultural norms around aging, participatory research practices and balancing autonomy with safety concerns for older adults in later life. This workshop brings together researchers and practitioners to discuss (1) opportunities and challenges for designing technological systems for caregiving for older adults; (2) longitudinal interactions with these systems as older adults progress through stages of functional and cognitive changes; (3) potential for such systems to support caregivers while centering older adults' privacy and autonomy needs; and (4) the influence of cultural norms on caregiving and technology use.

Designing explainable and personalized AI systems to provide support to older adults aging in place requires an understanding of their motivations and expectations for the explanations. This poster presents our ongoing work in exploring explanation preferences within AI systems for older adults aging in place with their carepartners. We do so by leveraging the speculative and iterative benefits of the Research through Design (RtD) approach in HCI, and explore variations in explanation requirements for different users by understanding their needs, goals and motivations for the different sources of information within the home. We illustrate an example for employing a Research through Design inquiry for the design of AI applications, adopting speculative methods to probe into future possibilities of Explainable AI (XAI) using a human-centered design framework. Through a Speed Dating study and a Co-Design activity, we investigate different explanation types and scenarios and argue for a shift in the algorithmic focus of Explainable AI research toward user-centered requirements, positioning explanation as a collaborative process between AI systems and users.

As the permeability of AI systems in interpersonal domains like the home expands, their technical capabilities of generating explanations are required to be aligned with user expectations for transparency and reasoning. This paper presents insights from our ongoing work in understanding the effectiveness of explanations in Conversational AI systems for older adults aging in place and their family caregivers. We argue that in collaborative and multi-user environments like the home, AI systems will make recommendations based on a host of information sources to generate explanations. These sources may be more or less salient based on user mental models of the system and the specific task. We highlight the need for cross technological collaboration between AI systems and other available sources of information in the home to generate multiple explanations for a single user query. Through example scenarios in a caregiving home setting, this paper provides an initial framework for categorizing these sources and informing a potential design space for AI explanations surrounding everyday tasks in the home.

While consumer digital calendars are widely used for appointment reminders, they do not fulfill all of the compensatory functions that are supported by calendars designed for cognitive rehabilitation therapies (CRTs). To inform the development of digital compensatory solutions, we employed a Distributed Cognition framework to elucidate how older adults with mild cognitive impairment (MCI) and their care partners manage calendaring details when supported by a traditional rehabilitation calendar. Participants mapped out their calendaring cognitive systems, composed of people and artifacts, completed a chart detailing how they track specific types of information, and shared calendaring strategies with each other. We used a Distributed Cognition framing to articulate information flows and breakdowns in participants’ calendaring systems, and we identified groups of participants with similar breakdowns in their calendaring systems. We close by suggesting design recommendations for digital calendaring approaches to support dyads of older adults with MCI and their care partners.

While commercial conversational agents (CA) (i.e. Google assistant, Siri, Alexa) are widely used, these systems have limitations in error-handling, flexibility, personalization and overall dialogue management that are amplified in care coordination settings. In this paper, we synthesize and articulate these limitations through quantitative and qualitative analysis of 56 older adults interacting with a commercial CA deployed in their home for a 10 week period. We look at the CA as a compensatory technology in an older adult's care network. We argue that the CA limitations are rooted in the rigid cue-and-response style of task-oriented interactions common in CAs. We then propose a redesign for CA conversation flow to favor flexibility and personalization that is nonetheless viable within the limitations of current AI and machine learning technologies. We explore design tradeoffs to better support the usability needs of older adults compared to current design optimizations driven by efficiency and privacy goals.

Visual question answering (VQA) lies at the intersection of language and vision research. It functions as a building block for multimodal conversational AI and serves as a testbed for assessing a model’s capability for open-domain scene understanding. While progress in this area was initially accelerated with the 2015 release of the popular and large dataset "VQA", new datasets are required to continue this research momentum. For example, the 2019 Outside Knowledge VQA dataset "OKVQA" extends VQA by adding more challenging questions that require complex, factual, and commonsense knowledge. However, in our analysis, we found that 41.4% of the dataset needed to be corrected and 10.6% needed to be removed. This paper describes the analysis, corrections, and removals completed and presents a new dataset: OK-VQA Version 2.0. To gain insights into the impact of the changes on OK-VQA research, the paper presents results on state-of-the-art models retrained with this new dataset. The side-by-side comparisons show that one method in particular, Knowledge Augmented Transformer for Vision-and-Language, extends its relative lead over competing methods. The dataset is available online. 1

As Conversational AI systems evolve, their user base widens to encompass individuals with varying cognitive abilities, including older adults facing cognitive challenges like Mild Cognitive Impairment (MCI). Current systems, like smart speakers, struggle to provide effective explanations for their decisions or responses. This paper argues that the expectations and requirements for AI explanations for older adults with MCI differ significantly from conventional Explainable AI (XAI) research goals. Drawing from our ongoing research involving older adults with MCI and their interactions with the Google Home Hub, we highlight breakdowns in conversational flow when older adults seek explanations. Based on our experience, we conclude with recommendations for HCI researchers to adopt a more human-centered approach as we move towards developing the next generation of AI systems.

In the Spring of 2020, closures and safe distancing orders swept much of the United States due to the COVID-19 pandemic. This paper presents a case study of pivoting an in-person empowerment program focused on lifestyle interventions for people newly diagnosed with Mild Cognitive Impairment (MCI) to an online program. Working as rapidly as possible to sustain participant engagement, our design decisions and subsequent iterations point to initial constraints in telehealth capabilities, as well as learning on the fly as new capabilities and requirements emerged. We present the discovery of emergent practices by family members and healthcare providers to meet the new requirements for successful online engagement. For some participants, the online program led to greater opportunities for empowerment while others were hampered by the lack of in-person program support. Providers experienced a sharp learning curve and likewise missed the benefits of in-person interaction, but also discovered new benefits of online collaboration. This work lends insights and potential new avenues for understanding how lifestyle interventions can empower people with MCI and the role of technology in that process.

Healthcare and wellbeing are two main interconnected application areas of conversational agents (CAs). There is a significant increase in research, development, and commercial implementations in this area. In parallel to the increasing interest, new challenges in designing and evaluating CAs have emerged. This study aims to identify key design, development, and evaluation challenges of CAs in healthcare and wellbeing research. The focus is on the very recent projects with their emerging challenges. A review study was conducted with 17 invited studies, most of which were presented at the ACM CHI2020 conference workshop on CAs for health and wellbeing. Eligibility criteria required the studies to involve a CA applied to a health or wellbeing project in an ongoing or recently finished project. The participating studies were asked to report on their projects' design and evaluation challenges. We used thematic analysis to review the studies. The findings include a range of topics from primary care to caring for older adults to health coaching. We identified four major themes: i) domain information and integration, ii) user-system interaction and partnership, iii) evaluation, and iv) conversational competence. While some challenges are shared with other CA application areas, safety and privacy remain the major challenges in the healthcare and wellbeing domains. An increased level of collaboration across different institutions and entities may be a promising direction to address some of the major challenges which otherwise would be too complex to be addressed by the projects with their limited scope and budget.

Improving medication management for older adults with Mild Cognitive Impairment (MCI) requires designing systems that support functional independence and provide compensatory strategies as their abilities change. Traditional medication management interventions emphasize forming new habits alongside the traditional path of learning to use new technologies. In this study, we navigate designing for older adults with gradual cognitive decline by creating a conversational “check-in” system for routine medication management. We present the design of MATCHA - Medication Action To Check-In for Health Application, informed by exploratory focus groups and design sessions conducted with older adults with MCI and their caregivers, alongside our evaluation based on a two-phased deployment period of 20 weeks. Our results indicate that a conversational “check-in” medication management assistant increased system acceptance while also potentially decreasing the likelihood of accidental over-medication, a common concern for older adults dealing with MCI.

Background Health care and well-being are 2 main interconnected application areas of conversational agents (CAs). There is a significant increase in research, development, and commercial implementations in this area. In parallel to the increasing interest, new challenges in designing and evaluating CAs have emerged. Objective This study aims to identify key design, development, and evaluation challenges of CAs in health care and well-being research. The focus is on the very recent projects with their emerging challenges. Methods A review study was conducted with 17 invited studies, most of which were presented at the ACM (Association for Computing Machinery) CHI 2020 conference workshop on CAs for health and well-being. Eligibility criteria required the studies to involve a CA applied to a health or well-being project (ongoing or recently finished). The participating studies were asked to report on their projects’ design and evaluation challenges. We used thematic analysis to review the studies. Results The findings include a range of topics from primary care to caring for older adults to health coaching. We identified 4 major themes: (1) Domain Information and Integration, (2) User-System Interaction and Partnership, (3) Evaluation, and (4) Conversational Competence. Conclusions CAs proved their worth during the pandemic as health screening tools, and are expected to stay to further support various health care domains, especially personal health care. Growth in investment in CAs also shows the value as a personal assistant. Our study shows that while some challenges are shared with other CA application areas, safety and privacy remain the major challenges in the health care and well-being domains. An increased level of collaboration across different institutions and entities may be a promising direction to address some of the major challenges that otherwise would be too complex to be addressed by the projects with their limited scope and budget.

In the Spring of 2020, COVID-19 closures and safe distancing orders required healthcare programs across the US to cease in-person treatment. This paper presents a case study of rapidly pivoting a novel, 12-month comprehensive clinical lifestyle program combining education, occupational therapy, cognitive training, and social interaction to an online application-based education program. The focus of the program is empowerment research for people newly diagnosed with mild cognitive impairment (MCI) and their care partners, and is conducted by the Emory Brain Health Center. Georgia Tech developed an education application (named MyCEP) for use with our MCI and care partner population combining off-the-shelf services and customized user interfaces. We used an iterative design and development process, testing our application with our end users and our treatment providers, and made updates based on our discovery of the need for new capabilities and requirements. We present the discovery of emergent practices by family members and healthcare providers to meet the new requirements for successful virtual engagement.

Conversational agents (CAs) such as Google Home or Alexa offer empowering opportunities for dyads composed of older adults with mild cognitive impairment (MCI) and their care partners. CAs support coordination and planning between the two, and can amplify the support that the care partner needs to provide. In this study, we observed how ten such dyads interacted with a Google Home over 10 weeks. We logged and analyzed 3,878 total interactions, interviewed the dyads to better understand their experiences, and also surveyed their individual preferences and priorities for automated assistance in the home. We found that CAs empowered both the people who had MCI, and their care partners. We observed that the utility of the CA in the day-to-day lives of users largely depended on how much the care partner scaffolded promising functionality, setting it up and contextualizing it for specific needs and desires.

Hearing aids help overcome the challenges associated with hearing loss, and thus greatly benefit and improve the lives of those living with hearing-impairment. Unfortunately, there is a lack of adoption of hearing aids among those that can benefit from hearing aids. Hearing researchers and audiologists are trying to address this problem through their research. However, the current proprietary hearing aid market makes it difficult for academic researchers to translate their findings into commercial use. In order to abridge this gap and accelerate research in hearing health care, we present the design and implementation of the Open Speech Platform (OSP), which consists of a co-design of open-source hardware and software. The hardware meets the industry standards and enables researchers to conduct experiments in the field. The software is designed with a systematic and modular approach to standardize algorithm implementation and simplify user interface development. We evaluate the performance of OSP regarding both its hardware and software, as well as demonstrate its usefulness via a self-fitting study involving human participants.

Hearing loss is one of the most common conditions affecting older adults worldwide. Frequent complaints from the users of modern hearing aids include poor speech intelligibility in noisy environments and high cost, among other issues. However, the signal processing and audiological research needed to address these problems has long been hampered by proprietary development systems, underpowered embedded processors, and the difficulty of performing tests in real-world acoustical environments. To facilitate existing research in hearing healthcare and enable new investigations beyond what is currently possible, we have developed a modern, open-source hearing research platform, Open Speech Platform (OSP). This paper presents the system design of the complete OSP wearable platform, from hardware through firmware and software to user applications. The platform provides a complete suite of basic and advanced hearing aid features which can be adapted by researchers. It serves web apps directly from a hotspot on the wearable hardware, enabling users and researchers to control the system in real time. In addition, it can simultaneously acquire high-quality electroencephalography (EEG) or other electrophysiological signals closely synchronized to the audio. All of these features are provided in a wearable form factor with enough battery life for hours of operation in the field.

Open Speech Platform (OSP) for hearing aids (HA) research comprises a realtimemaster hearing aid (RT-MHA); and an embedded web server (EWS) that serves webpages to any browser enabled device for monitoring and controlling RT-MHA. In this contribution, we present 4 classes of web-apps that can be combined and extended in novel ways to conduct psychophysical investigations beyond what is currently possible. (1) The Researcher apps provide access to all RT-MHA parameters; these settings can be saved in named files and recalled easily. (2) HA Fitting apps are written to capture audiologists’ hypotheses on HA parameters and their interactions in improving performance; conversely, they can incorporate human-in-the-loop research wherein, the user is forced to select one of two HA parameters (say, A and B) used to process specific speech stimuli stored in a database on OSP; the settings A and B are successively refined based on the user’s choice. (3) Ecological Momentary Assessment (EMA) apps are used to capture the user’s state for a given HA settings in a given listening environment. (4) Assessment apps enable various tests (e.g., syllable and word recognition tests) in a repeatable manner using stimuli stored in a database on OSP.