Initialization of Q-values by fuzzy rules for accelerating Q-learning

We demonstrate that Q-learning can be accelerated by appropriately specifying initial Q-values using fuzzy rules. Fuzzy rule-based Q-learning is fast but unstable. On the other hand, the conventional Q-learning is not fast while it has the theoretical convergence property. In our approach, advantages of both algorithms are combined into a single hybrid algorithm where the fuzzy rule-based Q-learning is first employed for specifying initial Q-values for the conventional Q-learning. The conventional Q-learning with appropriately specified initial Q-values requires much less iterations for obtaining good results than that with uniformly or randomly specified initial values. We examine the performance of the fuzzy rule-based Q-learning, the conventional Q-learning and the hybrid algorithm by computer simulations on gridworld problems.