AiSee: An AssistiveWearable Device to Support Visually Impaired Grocery Shoppers

People with visual impairments (PVI) experience simple tasks, such as grocery shopping, to be an essential difficulty. Although the recent emergence of AI-technology has been dramatically improving visual recognition capabilities, the application to the daily life of PVI is still complex and erroneous. For example, image recognition engines require a clear shot of the targeted object and a contextual understanding of the information the user requires. In this paper, we aimed to understand the PVI's needs and their pain points in the task of identifying grocery items. Following a user-centered design process, we iteratively tailored a truly wearable assistive device in the shape of a bone-conduction headset to the needs of our target group. We then evaluated the performance of our prototype, showing a success rate of 80% in recognizing grocery items in a controlled environment. We conclude with a pilot deployment demonstrating how our device can support grocery shopping in-the-wild.