Improving Indoor Mobility of the Visually Impaired with Depth-Based Spatial Sound

We present a novel system to help visually impaired people to move efficiently and safely in indoor environments by mapping input from a depth camera to spatially localized auditory cues. We propose a set of context-specific cues which are suitable for use in systems that provide minimal audio feedback and hence reduce masking of natural sounds compared to the audio provided by general-purpose sense substitution devices. Using simple but effective heuristics for detecting the floor and the side walls, we propose auditory cues that encode information about the distances to walls, obstacles, the orientation of the corridor or room, and openings into corridors or rooms. But the key to our system is the use of a spatial sound engine that localizes the generated sounds in 3D. We evaluate our system, comparing with MeloSee. Our preliminary pilot study with ten blindfolded participants suggests that our system was more helpful for spotting smaller obstacles on the floor, though neither system had a significant edge in terms of walking speed or safety.