Impact of Device and Environment on Visual-Auditory Sensory Substitution: A Comprehensive Behavioral Analysis Using the vOICe Algorithm

Recent studies have revealed that visual-auditory sensory substitution devices (SSDs) can effectively convey visual information to visually impaired or blind individuals through sound. However, SSDs are still not widely available to the visually impaired and blind community. Addressing these challenges requires not only the development of efficient SSD algorithms but also the evaluation of the impact of SSDs on the devices and environments in which they are used. This study represents the first attempt to analyze the impact of the device or environment used for SSDs on users’ perceptual abilities. To achieve this goal, we developed an experimental procedure that involves both the training of the SSD algorithm and the changing environment and devices that receive the audio signal. Two user experiments were conducted and revealed that user perception is significantly affected by the device and environment used for SSDs. These findings underscore the importance of considering the effect of the device and environment in which it will be used when designing an SSD algorithm or training system.