Evaluation of Short-Range Depth Sonifications for Visual-to-Auditory Sensory Substitution

Visual-to-auditory sensory substitution devices convert visual information into sound and can provide valuable assistance for blind people. Recent iterations of these devices rely on depth sensors. Rules for converting depth into sound (i.e., the sonifications) are often designed arbitrarily, with no strong evidence for choosing one over another. The purpose of this article is to compare and understand the effectiveness of five depth sonifications in order to assist the design process of future visual-to-auditory systems for blind people, which rely on depth sensors. The frequency, amplitude, and reverberation of the sound as well as the repetition rate of short high-pitched sounds and the signal-to-noise ratio of a mixture between pure sound and noise are studied. We conducted positioning experiments with 28 sighted blindfolded participants. Stage 1 incorporates learning phases followed by depth estimation tasks. Stage 2 adds the additional challenge of azimuth estimation to the first stage's protocol. Stage 3 tests learning retention by incorporating a 10-min break before retesting depth estimation. The best depth estimates in stage 1 were obtained with the sound frequency and the repetition rate of beeps. In stage 2, the beep repetition rate yielded the best depth estimation, and no significant difference was observed for the azimuth estimation. Results of stage 3 showed that the beep repetition rate was the easiest sonification to memorize. Based on the statistical analysis of the results, we discuss the effectiveness of each sonification and compare with other studies that encode depth into sounds. Finally, we provide recommendations for the design of depth encoding.