Generating structure in sensory data through coordinated motor activity

Evidence from human and animal studies suggests that neural and cognitive development unfolds in the course of active exploration of the sensory environment. We argue that the statistical structure of sensory inputs depends critically on coordinated motor activity (Lungarella, and Pfeifer, 2001). We develop a set of statistical measures to objectively characterize streams of sensory data from an information theoretical perspective and apply these measures to data sets obtained by adopting different motor strategies. The robotic platform used in the presented experiments consist of a color CCD camera mounted on a 2 DOF pan-tilt unit. Camera images were captured using a standard frame grabber and saved and processed within Matlab. Motor commands instructing the pan and tilt servos to move to specified positions were issued via a serial line interface (rate 2/sec). The stimulus was a "color Mondrian" composed of small color patches of a broad range of colors. A typical experimental run resulted in continuous time series of thousands of images, acquired under constant illumination and spatially averaged to yield a resolution of 16 × 12 pixels, with one image each for the red, green and blue channels of the color camera. Three different motor strategies were used to move the camera: 1) "still": the camera was moved to a random location within the color stimulus and fixed there. 2) "random": the camera was moved at random within a defined region of the stimulus. 3) "foveation": the camera was controlled by a simple neural network to foveate on red patches within the stimulus array. In these preliminary experiments, high complexity of sensory data is the result of a coordinated motor strategy (foveation) and active selection of specific sensory patterns present in the environment. This selection process generates correlations among parts of the image that are the result of dynamic coupling between the robot and the stimulus. Such correlations may serve as a basis for neural plasticity and development.