A neural network solution for exploitation and exploration problems

A reinforcement learning (RL) architecture using artificial neural networks (ANN) is proposed. The proposed method applies not only the exploitation mechanism but also the exploration mechanism to a partially unstructured environment and generates actions with learning capability. The exploitation capability is achieved by a normal distribution function with mean denoting a best action in the current state. Variances of the distribution function modified by a three-layer ANN indicate the range of exploring. The weights of this ANN are modified by an evaluated internal reinforcement signal so as to accumulating learning experiences. Meanwhile, the other two of ANNs approximate policy selection functions and estimate the state evaluation, respectively. They learn from by a reinforcement learning method. The performance of controlling a pendulum system is compared to one of the adaptive heuristic critic (AHC). The results show that the performance of proposed method is better than the AHC's.