A new random search method for neural networks learning-random search with variable search length (RasVal)

In this paper, a new random search method RasVal for neural networks (NN) learning is proposed. RasVal (random search with variable search length) is a kind of random search and it can find a global minimum instead of a local minimum using the capability of intensified and diversified searches. The main different point of RasVal from commonly used random search methods (RSM) is that the shape of the probability density function for random searching can be adjusted based on the information of success or failure of the search. First, RasVal is described and after that, performance between RasVal, backpropagation method (BP) and backpropagation method with momentum (Mom.BP) are compared. The performance is evaluated by the simulations which include both static and dynamic neural networks (NN) learning problems. In the simulations, NN is trained to realize nonlinear functions and to control a nonlinear crane system by using RasVal, BP and Mom.BP. Simulation results show that RasVal is superior or nearly equal to BP and Mom.BP because of the ability of intensification and diversification of the search.