Optimal Developmental Learning for Multisensory and Multi-Teaching Modalities

Most intelligent systems require the use of multiple sensors and motors, but training these systems becomes costly as the number and complexity of sensors and motors increase. Reinforcement learning promises to make the training process easier on human trainers but can take significantly more time. While many learning systems are either directly supervised or supervised through reinforcement, few can learn through multiple teaching modalities. Furthermore, many neural network systems do not allow for frame-incremental learning. This work introduces a real-time, frame-incremental framework that uses multiple sensors and multiple teaching modes (motor-supervision, reinforcement, or self-practice). The Developmental Network (DN) used in this work is optimal in the sense of maximum likelihood throughout a “lifetime”, under three conditions: (1) an incremental learning framework, (2) a training experience and (3) a limited amount of computational resources. Because the DN is free from the local minima problem, all DNs are performance equivalent and we record “lifetime” errors of a single trained network, removing a need for post-selection–post-selecting the luckiest network from many randomly initialized and trained neural networks according to their performances on test sets.