Generative AI (GenAI) tools are increasingly used for spreadsheet tasks, yet little is known about how blind users verify their outputs in accuracy-critical contexts. We conducted a study with 12 blind spreadsheet users to explore verification practices across tasks such as information extraction, formula generation, trend analysis, chart creation, and formatting. Participants never fully trusted outputs without verification and employed diverse strategies, including manual checks with screen reader and spreadsheet features, same AI-assisted verification, cross-AI tool validation, leveraging prior knowledge, and human assistance. These approaches were adapted based on task context, perceived risk, and users’ expertise. Errors were common, particularly in chart generation and formatting, some detected, others overlooked. While verification improved confidence, it was often effortful, time-consuming, or infeasible for visual tasks. We discuss how blind users utilize GenAI not only as a task performer but also as a verification aid and validator, highlighting design opportunities for more accessible and reliable spreadsheet use.
Combining conversational AI with refreshable tactile displays (RTDs) offers significant potential for creating accessible data visualization for people who are blind or have low vision (BLV). To support researchers and developers building accessible data visualizations with RTDs, we present a multimodal data interaction architecture along with an open-source reference implementation. Our system is the first to combine touch input with a conversational agent on an RTD, enabling deictic queries that fuse touch context with spoken language, such as "what is the trend between these points?" The architecture addresses key technical challenges, including touch sensing on RTDs, visual-to-tactile encoding, integrating touch context with conversational AI, and synchronizing multimodal output. Our contributions are twofold: (1) a technical architecture integrating RTD hardware, external touch sensing, and conversational AI to enable multimodal data interaction; and (2) an open-source reference implementation demonstrating its feasibility. This work provides a technical foundation to support future research in multimodal accessible data visualization.
Tactile graphics are widely used to present maps and statistical diagrams to blind and low vision (BLV) people, with accessibility guidelines recommending their use for graphics where spatial relationships are important. Their use is expected to grow with the advent of commodity refreshable tactile displays. However, in stark contrast to visual information graphics, we lack a clear understanding of the benefts that well-designed tactile information graphics offer over text descriptions for BLV people. To address this gap, we introduce a framework considering the three components of encoding, perception and cognition to examine the known benefts for visual information graphics and explore their applicability to tactile information graphics. This work establishes a preliminary theoretical foundation for the tactile-frst design of information graphics and identifes future research avenues.
Combining refreshable tactile displays (RTDs) with conversational AI offers a promising approach to accessible data visualization for people who are blind or have low vision (BLV). However, it remains an open question how these modalities should be integrated to support accessible data experiences. We address this through a co-design process with three BLV co-designers. Building on our prior Wizard-of-Oz study, we created a conversational tactile data interface (CTDI) that combines an RTD with an LLM-powered conversational agent, refined through four workshops over eight months. In addition to the resulting system, Graphy, we contribute design knowledge and recommendations for CTDIs. Co-designers used touch as the primary sensemaking channel for spatial understanding of the data's shape, trends, and relationships, reserved the agent for what touch could not resolve (e.g., calculation and analysis), and used the chart on the RTD to verify the agent's responses. Key findings include: a layered presentation that scaffolds chart exploration through progressive, interactive layers; a feedback grammar that distinguishes user- and agent-initiated tactile feedback; and a sequential interaction pattern – select, confirm, ask, verify – where each step grounds the last.
Data increasingly shape key aspects of everyday life, including education, health, and civic engagement. This special issue envisions a shift from traditional notions of accessibility toward a broader concept of inclusive data experiences, encompassing how people encounter, interpret, interact with, and act on data across diverse abilities and contexts. Moving beyond a primary focus on visual representations, it invites expanded attention to how diverse populations experience and engage with data in a wide range of settings. By bringing together urgent needs, innovative technical solutions, and visionary perspectives, this special issue aims to promote equitable access to the power and benefits of data.
Our work aims to develop new assistive technologies that enable blind or low vision (BLV) people to explore and analyze data readily. At present, barriers exist for BLV people to explore and analyze data, restricting access to government, health and personal data, and limiting employment opportunities. This work explores the co-design and development of an innovative system to support data access, with a focus on the use of refreshable tactile displays (RTDs) and conversational agents. The envisaged system will use a combination of tactile graphics and speech to communicate with BLV users, and proactively assist with data analysis tasks. As well as addressing significant equity gaps, our work expects to produce innovations in assistive technology, multimodal interfaces, dialogue systems, and natural language understanding and generation.
Productivity applications including word processors, spreadsheets, and presentation tools are crucial in work, education, and personal settings. Blind users typically access these tools via screen readers (SRs) and face significant accessibility and usability challenges. Recent advancements in Generative AI (GenAI) may address these challenges by enabling natural language interactions and contextual task understanding. However, there is limited understanding of SR users’ needs and attitudes toward GenAI assistance in these applications. We surveyed 99 SR users to gain a holistic understanding of the challenges they face when using productivity applications, the impact of these challenges on their productivity and independence, and their initial perceptions of AI assistance. Driven by their enthusiasm, we conducted interviews with 16 SR users to explore their attitudes toward GenAI and its potential usefulness in productivity applications. Our findings highlight its need to support existing SR workflows and the importance of enabling customization and task verification.
3D-printed models are increasingly used to provide people who are blind or have low vision (BLV) with access to maps, educational materials, and museum exhibits. Recent research has explored interactive 3D-printed models (I3Ms) that integrate touch gestures, conversational dialogue, and haptic vibratory feedback to create more engaging interfaces. Prior research with sighted people has found that imbuing machines with human-like behaviours, i.e., embodying them, can make them appear more lifelike, increasing social perception and presence. Such embodiment can increase engagement and trust. This work presents the first exploration into the design of embodied I3Ms and their impact on BLV engagement and trust. In a controlled study with 12 BLV participants, we found that I3Ms using specific embodiment design factors, such as haptic vibratory and embodied personified voices, led to an increased sense of liveliness and embodiment, as well as engagement, but had mixed impact on trust.
3D printing is a mainstream technology enabling the affordable production of 3D models that may enhance access and understanding of graphics for students who are blind or have low vision (BLV). However, the potential usefulness of a new technology does not guarantee its adoption. This paper presents a case study in the adoption of 3D printing as an accessible format for BLV education in Australia and New Zealand. Over the last six years, a community-driven research project engaged in awareness raising, created a community of practice and developed guidelines for the use of 3D printing in education. We evaluate the success of the project using an Implementation Science lens with the RE-AIM framework and identify the key factors for successful adoption. We hope this work will guide the adoption of 3D printing for BLV students and serve as an exemplar for the adoption of other assistive technologies.
3D printed models (3DPMs) are increasingly used to support the education of students who are blind or have low vision (BLV). As 3DPMs are more widely-adopted, educators are using more complex multi-part models. However, with this increased complexity comes additional challenges for their use, such as supporting audio labels of multiple parts as well as guiding the assembly and disassembly of the model. This work explores the co-design and evaluation of a system that supports the use of multi-part 3DPMs by BLV students. Working with BLV adults and children, as well as educators, an iPad application was developed to support interaction with an insect model, including speech interaction and support for assembly. Evaluation showed that the system was strongly enjoyed by students and educators were enthusiastic as they believed it would increase classroom engagement and inclusion, and its support for voice annotation could be used for assessment.
Cerebral Visual Impairment (CVI) is the set to be the leading cause of vision impairment, yet remains underrepresented in assistive technology research. Unlike ocular conditions, CVI affects higher-order visual processing-impacting object recognition, facial perception, and attention in complex environments. This paper presents a co-design study with two adults with CVI investigating how smart glasses, i.e. head-mounted extended reality displays, can support understanding and interaction with the immediate environment. Guided by the Double Diamond design framework, we conducted a two-week diary study, two ideation workshops, and ten iterative development sessions using the Apple Vision Pro. Our findings demonstrate that smart glasses can meaningfully address key challenges in locating objects, reading text, recognising people, engaging in conversations, and managing sensory stress. With the rapid advancement of smart glasses and increasing recognition of CVI as a distinct form of vision impairment, this research addresses a timely and under-explored intersection of technology and need.
In the current era of digitisation, a substantial amount of data significantly influences our daily experiences and decision-making processes. This data is systematically acquired from routine social interactions, online engagements, and the numerous sensors seamlessly integrated into our surroundings. Navigating this data-rich environment poses notable challenges, particularly when immersed in mixed reality environments. Situated Analytics (SA) was introduced to offer analytical reasoning within the physical space, utilizing visual analytics reasoning techniques combined with augmented reality to enhance the physical environment with real-time contextual information. The distinctive feature of SA lies in its contextual and situational, two-way, interactive visualisation, supported by a system and knowledge analytics to optimize the decision-making process. This paper discusses in-depth the components, processes, space, and mantra of SA, listing its characteristics and addressing the challenges associated with it.
Cultural institutions often employ commodity technology solutions for the provision of information in exhibitions. While these solutions provide a pragmatic approach, the efficacy, accessibility and user experience for people who are blind or have low vision (BLV) remains understudied. We conducted a case study with an Australian regional gallery to investigate BLV vistors’ use of commodity solutions – QR and NFC technology, voice recording push buttons, and touch-triggered electronics – and their efficacy in supporting the provision of accessible information. Through in-situ evaluation, post-visit questionnaire, and semi-structured interviews we present a deeper understanding of the needs and preferences of BLV visitors. Our findings illuminate key considerations to guide the meaningful use of commodity technologies by cultural institutions for accessibility. We hope that this case study will lead to more accessible solutions in cultural spaces, ensuring they more closely meet the needs of the BLV community and encourage more widely-adopted solutions.
Despite the recent surge of research efforts to make data visualizations accessible to people who are blind or have low vision (BLV), how to support BLV people's data analysis remains an important and challenging question. As refreshable tactile displays (RTDs) become cheaper and conversational agents continue to improve, their combination provides a promising approach to support BLV people's interactive data exploration and analysis. To understand how BLV people would use and react to a system combining an RTD with a conversational agent, we conducted a Wizard-of-Oz study with 11 BLV participants, where they interacted with line charts, bar charts, and isarithmic maps. Our analysis of participants' interactions led to the identification of nine distinct patterns. We also learned that the choice of modalities depended on the type of task and prior experience with tactile graphics, and that participants strongly preferred the combination of RTD and speech to a single modality. In addition, participants with more tactile experience described how tactile images facilitated a deeper engagement with the data and supported independent interpretation. Our findings will inform the design of interfaces for such interactive mixed-modality systems.
Spreadsheets are widely used for storing, manipulating, analyzing, and visualizing data. Features such as conditional formatting, formulas, sorting, and filtering play an important role when understanding and analyzing data in spreadsheets. They employ visual cues, but we have little understanding of the experiences of blind screen reader (SR) users with such features. We conducted a study with 12 blind SR users to gain insights into their challenges, workarounds, and strategies in understanding and extracting information from a spreadsheet consisting of multiple tables that incorporated data analysis features. We identified five factors that impact blind SR users’ experiences: cognitive overload, time-information trade-off, lack of awareness and expertise, inadequate system feedback, and delayed and absent SR responses. Drawn from these findings, we discuss design suggestions and future research agenda to improve SR users’ spreadsheet experiences.
Over the past decade, considerable research has investigated Vision-Based Assistive Technologies (VBAT) to support people with vision impairments to understand and interact with their immediate environment using machine learning, computer vision, image enhancement, and/or augmented/virtual reality. However, this has almost totally overlooked a growing demographic: people with Cerebral Visual Impairment (CVI). Unlike ocular vision impairments, CVI arises from damage to the brain’s visual processing centres. Through a scoping review, this paper reveals a significant research gap in addressing the needs of this demographic. Three focus studies involving 7 participants with CVI explored the challenges, current strategies, and opportunities for VBAT. We also discussed the assistive technology needs of people with CVI compared with ocular low vision. Our findings highlight the opportunity for the Human-Computer Interaction and Assistive Technologies research community to explore and address this underrepresented domain, thereby enhancing the quality of life for people with CVI.
This paper introduces the use of Augmented Reality as an immersive analytical tool in the physical world. We present Situated Analytics, a novel combination of real-time interaction and visualization techniques that allows exploration and analysis of ...
Over the past decade, considerable research has been directed towards assistive technologies to support people with vision impairments using machine learning, computer vision, image enhancement, and/or augmented/virtual reality. However, this has almost totally overlooked a growing demographic: people with Cerebral Visual Impairment (CVI). Unlike Ocular Vision Impairments (OVI), CVI arises from damage to the brain’s visual processing centres. This paper introduces CVI and reveals a wide research gap in addressing the needs of this demographic. Through a scoping review, we identified 14 papers at the intersection of these technologies and CVI. Of these, only three papers described assistive technologies focused on people living with CVI, with the others focusing on diagnosis, understanding, simulation or rehabilitation. Our findings highlight the opportunity for the Human-Computer Interaction and Assistive Technologies research community to explore and address this underrepresented domain, thereby enhancing the quality of life for people with CVI.
Refreshable tactile displays (RTDs) are predicted to soon become a viable option for the provision of accessible graphics for people who are blind or have low vision (BLV). This new technology for the tactile display of braille and graphics, usually using raised pins, makes it easier to generate and access a large number of graphics. However, it differs from existing tactile graphics in terms of scale, height and fidelity. Here, we share the perspectives of four key stakeholders -- blind touch readers, vision specialist teachers, accessible format producers and assistive technology providers -- to explore the potential uses, advantages and needs relating to the introduction of RTDs. We also provide advice on what role the data visualisation community can take to help ensure that people who are BLV are best able to benefit from the introduction of affordable RTDs.
Whilst the visual arts is a very practical and tangible field, data visualisation is a combination of programming skills, data science and design theory. Each of these can be taught in a variety of ways, from traditional methods of lectures to technical programming classes or even asynchronous worksheets. The challenge is how to cover all of the required content within the constraints of the teaching period, environment, delivery method and ensuring the appropriate activities and assessment tasks. In this paper we reflect on the past 8 years of delivering the postgraduate unit ‘Data Exploration and Visualisation’ at Monash University. We present the four different ways that the same content has been taught to different cohorts, through a combination of on-campus lectures, large-format workshops, small group tutorials and online or hybrid tutorials and workshops, as well as various readings, videos and asynchronous activities. We explain the different external and internal requirements set for the teaching and describe the adaption of the teaching methods and assessments in order to meet these conditions. We reflect on our experiences of teaching multiple methods across different formats from very small to very large student cohorts.
Peter Stuckey合作论文数Faculty of Information Technology, Monash University47
Mark Wallace合作论文数Monash University,Faculty of Information Technology6
Martin Odersky合作论文数School of Computer and Communication Sciences, Swiss Federal Institute of Technology in Lausanne4