Qualitative Adaptive Critics

In this paper we compare the use of qualitative adaptive critics to traditional quantitative critics for the design of control systems. Our approach uses a qualitative implementation of the Bellman recursion to train critic and controller networks. This extends previous work with univariate plants to multivariate plants with multiple control objectives. The results indicate that the superior control achieved by more sophisticated model based adaptive critic methods is due to qualitatively more accurate estimates of the gradient of the secondary utility function as opposed to increased numerical precision.