Although virtual reality (VR) was originally conceived of as a multi-sensory experience, most developers of the technology have focused on its visual aspects to the detriment of other senses such as hearing. This paper presents design patterns to make virtual reality fully accessible to non-visual users, including totally blind users, especially with non-verbal social interactions. Non-visual VR has been present in the blindness audio game community since the early 2000s, but the conventions from those interfaces have never been described to a sighted audience, outside of a few limited sonification interface papers. This paper presents non-visual design patterns created by five of the top English-speaking audio game developers through a three round Delphi method, encompassing 29 non-verbal social interactions grouped into 12 categories in VR, including movement, emotes, and self-expression. This paper will be useful to developers of VR experiences who wish to represent non-verbal social information to their users through non-visual conventions. These methods have only been rigorously tested through the commercial market, and not through scientific approaches. These design patterns can serve as the foundation for future investigation in exploring non-visual non-verbal social interactions in VR.

Topological features of time-dependent, three-dimensional (3D) vector flow fields, such as wall shear stress (WSS) fixed points, are considered surrogates of pathological blood flow dynamics in cardiovascular diseases. Fixed-point visualizations are typically constrained to two-dimensional (2D) spaces, yet they aim to display complex spatiotemporal (four-dimensional (4D)) dynamics. There is a need for visualization strategies to reduce occlusion and reliance on animations to allow the detection of holistic flow patterns. Using intracranial aneurysms as a use case, we present the fixed-point carousel, a novel approach to visually depicting the "4D" nature of WSS fixed points via (1) topographic mapping of the 3D aneurysm sac to overcome occlusion while preserving fixed-point distances and sac morphological features; and (2) arranging these into a carousel model to present with temporal dimension holistically. Examples are presented for image-based computational fluid dynamic (CFD) models of intracranial aneurysms, illuminating the intricate and distinct fixed-point trajectories and interactions, a necessary step toward understanding the volumetric flow manifolds that drive them for this and other cardiovascular-and potentially nonbiomedical-fluid dynamics applications.

Web Content Accessibility Guidelines (WCAG) require digital diagrams to be tagged with screen readable text descriptions for access by blind and partially sighted individuals (BPSI). The aim of these guidelines is to comply with human rights-based accessibility legislation, which aims to preserve the normative agency of BPSI (the ability to reflect on, evaluate and act upon a conception of what constitutes a worthwhile life for themselves). However, theories from the Diagrams community suggest that text and diagrams offer distinctly different constraints. For example, Shimojima’s Constraint Hypothesis (the relationship of structural constraints to target problem constraints in a mode of representation establish the variance of inferential potential) and the interrelated free ride phenomenon (additional inferences can be made in a representation if the relationship of constraints is a good match). Therefore, a guideline that requires the text description of a diagram (such as via the WCAG) might limit the normative agency diagram users who are BPSI. Despite the apparent necessity of providing non-visual alternatives of diagrammatic properties for accessibility, they are rarely explored or developed sufficiently to be consistently provided to BPSI. Thus, we argue that the affordances of diagrammatic representations provide possibilities for normative agency that are lost if not represented non-visually in diagrams designed for accessibility.

Everyday experience is multi-sensory, and user experience (UX) design aims to extend this to interactions with products, services, and designed worlds. However, tools and pedagogies for UX are overwhelmingly visual, whereas human-rights-based accessibility legislation mandates the inclusion of diverse peoples, including blind and partially sighted individuals. Coupling auditory and haptic UX techniques from human-computer interaction with industrial design’s (ID) cross-modal tradition of prototyping physical products fostered our novel cross-modal UX course for second-year ID undergraduates. Affordance-based theories of perception-action and Gestalt principles of perceptual organization were used to inform design in auditory, tactile, and visual sensory modalities situated in a novel pedagogical framework. Each week theoretical models were presented alongside hands-on workshops using the BBC micro:bit, developing computational literacy through cross-modal physical prototyping. Student projects demonstrate an understanding of theory and practice and include auditory and tactile interfaces.

Drawing as an activity aids problem solving, collaboration, and presentation in design, science, and engineering and artistic creativity as well as expression in the arts. Unfortunately, blind, and partially sighted learners still lack an inclusive and effective drawing tool, even in the digital age. In response, this research aims to explore what an effective drawing tool for blind and partially sighted individuals (BPSI) would be. Raised-line drawing kits aim to provide this, but in prior work, our usability tests of raised line graphics with blind and partially sighted participants rated the raised line graphics that we tested as barely comprehensible relative to 3D models, which they rated as highly comprehensible. Semi-structured interviews with our participants afterward suggest that they found 3D models to be more comprehensible because these are consistent with haptic principles of perception whereas conventions of raised line graphics, such as a line representing a surface edge, replicate visual cues of source images and thereby violate haptic principles of perception. Therefore, we hypothesize that a drawing tool for blind and partially sighted drawers could be effective by recruiting affordances of 3D models. Through co-design sessions conducted during the Covid-19 pandemic with blind and partially sighted drawers (BPSD), we prototyped a tangible 3D model construction kit for non-visual haptic drawing with a digital interface to a 3D virtual environment. Our current investigation of user needs is informing us of our ongoing iterative development of an accessible 3D scanning application that is enabling blind and partially sighted individuals to build and scan in 3D models constructed from a more flexible range of materials beyond what was possible with our previous prototype.

Purpose Communicating complex blood flow patterns generated from computational fluid dynamics (CFD) simulations to clinical audiences for the purposes of risk assessment or treatment planning is an ongoing challenge. While attempts have been made to develop new software tools for such clinical visualization of CFD data, these often overlook established medical imaging/visualization practice and data infrastructures. Here, leveraging the clinical ubiquity of the DICOM file format, we present techniques for the translation of CFD data to DICOM series, facilitating interactive visualization in standard radiological software. Methods Unstructured CFD data (volumetric fields of velocity magnitude, Q-criterion, and pathlines) are resampled to structured grids. Novel raster-based techniques that simulate experimental optical blurring are presented for bringing simulated pathlines into structured image volumes. DICOM series are created by strategically encoding these data into the file's PixelArray tag. Lumen surface information is also strategically encoded into a different range of pixel intensities, allowing hemodynamics and morphology to be co-visualized in a single volume using opacity-based rendering transfer functions. Results We show that 3D temporal CFD data represented as structured DICOM series can be rendered interactively in Horos, a widely-used medical imaging/radiology software. Our transfer function-based approach allows for representations of scalar isosurfaces, volumetric rendering, and tubular pathlines to be modified in real-time, resembling conventional unstructured visualizations. Careful selection of voxelization ROIs helps to ensure that data are kept lightweight for real-time rendering and minimal storage. Conclusion While our approach inherently sacrifices some of the advanced visualization capabilities of specialized software tools, we believe our closer consideration of standardization can help to facilitate meaningful clinical interaction. This work opens up possibilities for the complete integration of measured and simulated data in established radiological software environments and workflows from PACS storage to 3D/4D visualization.

Spatial skills are critical for understanding the relations among objects and people, playing an important role in how we interact with the world. Spatial relationships are built through interactions with physical objects; however, in computational/online environments, these change to bi-dimensional media and computer-assisted design comprised of 3D representations viewable through a flat screen. Due to spatial immersion and interaction limitations, a traditional 2D and 3D approach presents challenges to partially sighted, blind, and sighted individuals. This paper presents the prototyping of a co-design Augmented Reality (AR) authoring tool by recruiting inclusive emerging affordances of consumer-level AR technologies within the context of current e-learning provisions in subject matters, including inclusive design, engineering design, game hardware design, and health sciences. This work has been inspired by the COVID-19 pandemic that has shown the need to level the field in inclusive design for teaching a subject typically oriented to the sighted. Our prototype allows users to create e-learning content for visualization, interaction, collaboration, and inclusive learning. Future work will investigate our tool's impact on skills development and content creation.

This paper is associated with a poster winner of a 2020 American Physical Society's Division of Fluid Dynamics (DFD) Milton van Dyke Award for work presented at the DFD Gallery of Fluid Motion. The original poster is available online at the Gallery of Fluid Motion, https://doi.org/10.1103/APS.DFD.2020.GFM.P0004.

Everyday perception and action are fundamentally multisensory. Despite this, the sole reliance on visualization for the representation of complex 3D spatiotemporal data is still widespread. In the past we have proposed various prototypes for the sonification of dense data from computational fluid dynamics (CFD) simulations of turbulent-like blood flow, but did not robustly consider the perception and associated meaning-making of the resultant sounds. To reduce some of the complexities of these data for sonification, in this work we present a feature-based approach, applying ideas from auditory scene analysis to sonify different data features along perceptually-separable auditory streams. As there are many possible features in these dense data, we followed the analogy of “caricature” to guide our definition and subsequent amplification of unique spectral and fluctuating features, while effectively minimizing the features common between simulations. This approach may allow for better insight into the behavior of flow instabilities when compared to our previous sonifications and/or visualizations, and additionally we observed benefits when some redundancy was maintained between modalities.

More than a decade ago, the authors proposed establishing a basis for scientific exploration of blood-flow dynamics intertwined with the visual arts. Here they present a case study showing how paradigms they codeveloped for visually abstracting cerebral aneurysm blood flows were extrapolated to sonification and bimodal representations, and how a close interdisciplinary partnership was effected by guiding engineering students versed in the arts and artists adept with digital technology toward final outcomes greater than the sum of their parts.

According to the World Health Organization, there are more than one billion people with moderate or severe distance vision impairment or blindness worldwide in 2020. Due to recent COVID-19 safety measures, online events have moved to online platforms in order to maintain physical distancing. The transition towards online gatherings have shown gaps associated with technology and internet access, along with lacking accessibility, which has resulted in reactive responses to improve usability by adding captioning, compatibility with screen readers, and image captioning to cite some examples. However, traditional video conference lacks immersion and presence otherwise found in virtual reality (VR), for example Mozilla Hubs, Engage VR, AltspaceVR, and Virbela amongst others. Although these services and consumer-level hardware are becoming ubiquitous, there are still accessibility challenges to be met when designing VR experiences. In this work-in-progress paper we explore the design of an inclusive web-based VR application that simulates registering at a virtual conference and navigating to a presentation room. Our preliminary study focused on usability and engagement by inviting participants to experience the demo eyes closed. Our preliminary results show the audio cues were helpful to navigate the environment without visual feedback.

Non-visual maps have been primarily made as raised line graphics with braille, but recent research has suggested that 3D models and interactivity make maps more accessible and easier to understand for non-visual users. This paper describes the design and evaluation of an interactive 3D map, which is a scale model of an environment with objects on it that play audio labels when a user points to them. In this study, we designed the interactive map to make an existing playground, the Magical Bridge playground [1] in Palo Alto, California, more accessible to visually impaired visitors. While this inclusive playground was specifically “created to be highly accessible to all visitors and to remove the physical and social barriers of today’s typical playgrounds and give everyone a place to play,” it is still challenging for visually impaired visitors to navigate independently.

Informed by high-resolution computational fluid dynamics (CFD) simulations, we present a strategy using a temporal filtering approach to examine the three-dimensional structures of velocity fluctuations filtered based on the global spectral content. Research suggests the presence of transient, turbulent-like flow instabilities at a range of frequencies from 10 Hz up to 1 kHz, some of which are associated with clinical reports of aneurysm vibration or 'bruits', and which may promote aneurysm growth or rupture. To isolate and visualise these instabilities with respect to their frequency, the filtering technique presented in this work is applied to the flow simulations of three middle cerebral artery (MCA) aneurysms and three internal carotid artery (ICA) siphons. Vortex cores associated with the different frequency bands are then visualised together to highlight their spatiotemporal interactions. Inspired by visual styles of illustration, we present a rendering strategy depicted with outlines, silhouettes and two-tone cel-shading to prevent occlusion and emphasise the resulting flow structures of importance while the other details are given less weight to de-emphasise their presence in the background of the image plane. Reinforcing previous studies in the literature, the current work also confirmed the presence of flow fluctuations to the order of up to 1 kHz when modelled adequately using high-resolution CFD simulations.

We present our participatively and iteratively designed 3D audio-tactile globe that enables blind and low-vision users to perceive geo-spatial information. Blind and low-vision users rely on learning aids such as 2D-tactile graphics, braille maps and 3D models to learn about geography. We employed participatory design as an approach to prototyping and evaluating four different iterations of a cross-sensory globe that uses 3D detachable continents to provide geo-spatial haptic information in combination with audio labels. Informed by our participatory design and evaluation, we discuss cross-sensory educational aids as an alternative to visually-oriented globes. Our findings reveal affordances of 3D-tactile models for conveying concrete features of the Earth (such as varying elevations of landforms) and audio labels for conveying abstract categories about the Earth (such as continent names). We highlight the advantages of longitudinal participatory design that includes the lived experiences and DIY innovations of blind and lowvision users and makers.

This study evaluated a web-based auditory map prototype built utilizing conventions found in audio games and presents findings from a set of tasks participants performed with the prototype. The prototype allowed participants to use their own computer and screen reader, contrary to most studies, which restrict use to a single platform and a self-voicing feature (providing a voice that talks by default). There were three major findings from the tasks: the interface was extremely easy to learn and navigate, participants all had unique navigational styles and preferred using their own screen reader, and participants needed user interface features that made it easier to understand and answer questions about spatial properties and relationships. Participants gave an average task load score of 39 from the NASA Task Load Index and gave a confidence level of 46/100 for actually using the prototype to physically navigate.

Identifying auditory correlates of the graphic-linguistic distinction informs our design of an auditory display based on Charles Minard’s depiction of Napoleon’s Russia campaign – the gold standard for visual (graphic) information design and therefore a grand challenge for auditory display design. We identify viable alternatives to the text-only translations currently employed in making graphics accessible to blind and/or low-vision individuals by introducing sounds bearing strong ecological resemblances to Minard’s depictions. Our integration of theoretical work about classic distinctions with common properties across diagrammatic and auditory display communities reveals practical opportunities for designing inclusive and accessible graphics.