Autonomous mental development in high dimensional state and action spaces

Autonomous mental development (AMD) of robots opened a new paradigm for developing machine intelligence, using neural network type of techniques and it fundamentally changed the way an intelligent machine is developed from manual to autonomous. The work presented is a part of SAIL (self-organizing autonomous incremental learner) project which deals with autonomous development of entire humanoid robot with vision, audition, manipulation and locomotion. The major issue addressed is the challenge of high dimensional action space (5 to 10) in addition to the high dimensional context state space (hundreds to thousands and beyond), typically required by an AMD machine. This is the first work that studies a high dimensional (numeric) action space in conjunction with a high dimensional perception (context state) space, under the AMID mode. Two new learning algorithms, Direct Update on Direction Cosines (DUDC) and High-Dimensional Conjugate Gradient Search (HCGS), are developed, implemented and tested. The convergence properties of both the algorithms and their targeted applications are discussed. Autonomous learning of speech production under reinforcement learning is studied as an example.