Graphical User Interface and Architecture of an Android Mobile Application for a Visual-to-Auditory Sensory Substitution System

Visual-to-auditory sensory substitution systems use various algorithms to convert visual information into an auditory signal, thereby providing perception and interpretation of data that is traditionally perceived through vision. In order for a visual-to-auditory sensory substitution system to be mobile enough for navigation, the processing of visual information in such a system should be performed on a mobile device, for example, on a smartphone. This paper presents a graphical user interface (GUI) and architecture for an Android-based mobile application named SightSense for a visual-to-auditory sensory substitution system. The SightSense application converts binary (“black and white”) images from both built-in smartphone camera and external camera using three different algorithms of visual-to-auditory sensory substitution. The GUI was developed in accordance with the requirements of the international standard Web Content Accessibility Guidelines 2.2 and the principles of Google Material Design. For the software implementation of the mobile application Kotlin, which is the official development language for Android, was chosen.