GesturePod : Programmable Gesture Recognition for Augmenting Assistive Devices

UPDATED—May 31, 2018. We have designed a GesturePod that converts any white cane into an interactive cane that can enable easy access to smartphones and other home devices for people who are visually impaired (VI) and the elderly. Users can control devices by performing programmable gestures on the cane. For example, a user can answer an incoming call with a double swipe. Our GesturePod is a lightweight, non-intrusive device built from inexpensive Inertial Measurement Unit (IMU) sensors and a battery-powered Cortex M0+ microcontroller. A key technical contribution of this work is the development of a robust real-time gesture recognition system based on Machine Learning (ML) models that can be deployed on tiny microcontrollers. We also developed an Android App which pairs with GesturePod and makes it easier and faster to complete frequent phone tasks like answering a call or reading notifications. A study with 12 VI participants demonstrated that with just 10 minutes of training, users were able to perform gestures with >90% accuracy and complete some of the tasks 9 times faster. Author