A neural approach to adaptive behavior and multi-sensor action selection in a mobile device

Sampling multisensory information and taking the appropriate motor action is critical for a biological organism's survival, but a difficult task for robots. We present a Neurally Organized Mobile Adaptive Device (NOMAD), whose behavior is controlled by a simulated nervous system based on the anatomy and physiology of the vertebrate brain, that is capable of action selection in a real world environment. NOMAD's nervous system consists of an auditory system, a visual system, a taste system, sets of motor neurons capable of triggering behavior, a tracking system driven by visual stimuli, and a value system. The device itself, which moves autonomously, has a CCD camera for vision, microphones for hearing, and a gripper manipulator to pick up and taste objects by measuring the object's conductivity. Similar to a biological organism, NOMAD learns to categorize sensory information from its environment with no prior instruction, associate positive and negative value with this sensory information, and then learn to select the appropriate motor actions. We suggest that this neurobiological approach to action selection may be generalized to other robot systems.