Acquiring localization ability by interaction between motion and sensing

We propose a learning model of sound source localization through the interaction between motion and audio-visual sensing. Living organisms can estimate the direction of a sound source based on the biaural time difference between their right and left ears. If the sound velocity and the distance between the ears are given, we can easily find the direction of a sound source from the time difference. However, living organisms do not know these parameters or the geometrical relation between time difference and source direction. They acquire the ability to correctly localize a sound source by repeated sensing and head movements. The model consists of two modules-a visual estimation module consisting of a three-layer perceptron and an auditory estimation module consisting of a neural network with a look up table algorithm. We assume that the visual estimation module has learned the correct input-output relation and the auditory estimation module uses the output from the visual estimation module to learn. We conducted computer simulation to investigate the validity of the proposed module. The experimental results demonstrate that sound source localization ability can be acquired without supervisors for a nonlinear system, and the system can correctly localize a sound source even when it is out of sight.