Image/Video Summarization in Text/Speech for Visually Impaired People

In the year 2022, an estimated 2.2 billion people around the globe will have a visual impairment. The problem may be hereditary or due to accidents. Nonetheless, technological advancements in helping visually impaired people have been going on for a long time. The admittance of technical concepts such as robotics, Machine Learning, and Artificial Intelligence for societal needs has proven worthwhile. The blind or visually impaired people learn about their surroundings through other senses, such as touch, hearing, and smell. Our proposed work aims to build an end-to-end solution for visually impaired people to help them grasp the environment by summarizing the images or video streams with the help of Machine Learning paradigms. The proposed work uses a pre-trained Caffe Object Detection model and requires less data for training and detection. We have developed a Client-Server model for our proposed idea wherein the significant computations happen on the server side, which is the Object detection model, and the client App is developed using Android. The app also has a text-to-signal processing feature that helps summarize the objects detected in the form of an audio catalog.