Keep in Touch: Combining Touch Interaction with Thumb-to-Finger Gestures for People with Visual Impairment

We present a set of 8 thumb-to-finger microgestures (TTF mu Gestures) that can be used as an additional modality to enrich touch interaction in eyes-free situations. TTF mu Gestures possess characteristics especially suited for people with visual impairment (PVI). They have never been studied specifically for PVI to improve accessibility of touchscreen devices. We studied a set of 33 common TTF mu Gestures to determine which are feasible and usable without seeing while the index is touching a surface. We found that the constrained position of the hand and the absence of vision prevent participants from being able to efficiently target a specific phalanx. Thus, we propose a set of 8 TTF mu Gestures (6 taps, 2 swipes) balancing resiliency (i.e., low error-rate) and expressivity (i.e., number of possible inputs): as a dimension combined with the touch modality, it would realistically multiply the touch command space by eight. Within our set of 8 TTF mu Gestures, we chose a subset of 4 mu Gestures (2 taps and 2 swipes) and implemented an exploration scenario of an audio-tactile map with a raised-line overlay on a touchscreen and tested it with 7 PVI. Their feedback was positive on the potential benefits of TTF mu Gestures in enhancing the touch modality and supporting PVI interaction with touchscreen devices.