A Framework to Assess Cortical Activity of Visually Impaired Persons during Training with a Sensory Substitution Device

In this paper we describe a framework for assessing the cognitive and emotional activity of blind and visually impaired people in relation to the usage of a sensory substitution system (SSD). The overall objective is to aid the design and development of the SSD by understanding how the different choices in encoding and rendering the environmental information to the user, as well as training, affects the user perception and experience while using the system. To this end, a framework for EEG data acquisition and processing, tightly integrated with the SSD is proposed. The interpretation of the cortical activity of the visually impaired persons is performed in correlation with results from psychophysical tests, where accuracy and response times for various tasks are recorded. The framework allows for synchronized acquisition of EEG measurements and system/user events while training and testing with the SSD. The training and testing with the SSD is performed in simulated (virtual) environments. The hardware and software architecture of the framework is described. Preliminary results of the cortical activity analysis for blind users are presented to show case the exploitation of the proposed framework.