Investigating and modeling the effects of cognitive factors on multisensory perception using skin conductance

Audiovisual perception associated to a spatial localization task is investigated using a focused attention paradigm: in presence of both acoustic and visual stimuli, subjects are required to localize either one or the other of these stimuli. Behavioral measures (i.e. subjects' localization errors), as well as skin conductance responses (i.e., components of the orienting responses) are acquired during the experiment. The subjects' performance on the localization task differ depending on the sensory nature of the stimulus they have to localize. The processing of the incoming information and the mobilized cognitive resources also depends on the task, as suggested by the analysis of the skin conductance responses. The latter are incorporated to a bayesian network inferring the subjects' error given audiovisual stimulus positions. This amounts to provide the model with some knowledge about the cognitive factors that interface with multisensory perception (resulting in a different exploitation of similar audiovisual information). As a result, the subjects' performance are better estimated by the model.