A Sensory Substitution Device Inspired by the Human Visual System

The purpose of this paper is to introduce and initially evaluate an experimental sensory substitution device (SSD), which converts color to audible sound. Our system, called Colorophone provides continuous information about the color, light intensity and distance. The color sonification method is inspired by the human visual system. While designing sonification method we took into consideration mismatch in sensory bandwidth, possible sensory overload, nature of sensory spatiotemporal continuity and cross-modal correspondences between colors and sounds. We conducted a preliminary experimental evaluation of the color and object recognition abilities as well as orientation ability. We show that blindfolded participants can easily acquire the color information coded in the auditory stimuli generated by the Colorophone system after a short learning session. Our preliminary experiments indicate that sonified color information helps in object identification and enhances orientation ability. The Colorophone proved to be an easy to understand and intuitive visual-to-auditory coding system, which shows promising results in future visual rehabilitation of the blind.