TD based reinforcement learning using neural networks in control problems with continuous action space

While most of the research on reinforcement learning assumed a discrete control space, many of the real world control problems need to have continuous output. This can be achieved by using continuous mapping functions for the value and action functions of the reinforcement learning architecture. Two questions arise here however. One is what sort of function representation to use and the other is how to determine the amount of noise for search in action space. The ubiquitous back-propagation neural network is used here to learn the value and action functions. Next, the reinforcement predictor that is intended to predict the next reinforcement is introduced that also determines the amount of noise to add to the controller output. This proposed reinforcement learning architecture is found to have a sound online learning control performance through a computer simulation of the ball and beam system as an example plant.