Federated Learning for Obstacle Detection to Assist the Visually-Impaired Using Augmented Reality

Vision impairment increases risks such as social isolation, mobility challenges, and falls. Wearable Augmented Reality (AR) devices with Artificial Intelligence (AI) can enhance sensory perception by enabling real-time recognition of obstacles, assisting visually impaired individuals during street navigation, aiming to reduce the risk of falls. In this paper, we propose a Federated Learning (FL) framework for outdoor obstacle detection using resource-constrained edge AR devices. As such, our proposed framework is designed to optimize energy efficiency by partially fine-tuning a generic pre-trained model originally developed for visually impaired assistance. Our framework is evaluated on a testbed with NVIDIA Jetson Nano, demonstrating that FL improves accuracy over standalone models on each AR device, while achieving an accuracy comparable to centralized approaches, but without the need to transfer the local raw data on each AR device to a central server/cloud to alleviate privacy concerns.