From Vision to Sound: Enhanced Object Localization for Visually Impaired and Blind

According to the World Health Organization, hundreds of millions of people worldwide are affected by visual impairments. This has profound personal effects, as our perception, cognition, learning, and daily activities are mediated through vision. In this study, we introduce a wearable visual assistance system designed for visually impaired and blind individuals to help with locating personal items, an essential daily activity. Our system enhances object localization by bridging advanced computer vision-based object detection with spatial sound feedback. We run our method locally on an edge device to protect user privacy. We conducted extensive experiments with 44 participants to study the effectiveness of our system. We evaluated our system using the System Usability Scale and the NASA Task Load Index questionnaires. The experimental results show that our visual assistant system reduces the average object localization time by 37% and improves the successful localization rate by 2.2 times. The positive feedback from the participants highlights the potential of our system to improve the quality of life of visually impaired and blind people. We have made our source code publicly available at https://github.com/IS2AI/visual_assistant to stimulate further research in this area.