How To Discretize Continuous State-Action Spaces in Q-Learning: A Symbolic Control Approach

This paper addresses challenges in handling continuous state-action spaces using Q-learning. The novel Q learning algorithm generates two Q-tables that bound the Q-values of the continuous system. Theoretical analysis establishes their convergence and bounds the loss in the Q-values. The algorithm achieves optimality within desired accuracy, offering control over the trade-off between precision and computational complexity. Valuable insights for learning parameter selection and controller refinement are provided. The engineering relevance of the proposed Q-learning based symbolic model is illustrated through Mountain Car control problem.