Simultaneous identification and adaptive control of unknown systems over finite parameter sets

The problem considered is one of simultaneously identifying an unknown system while adequately controlling it. The system can be any fairly general discrete-time system and the cost criterion can be either of a discounted type or of a long-term average type, the chief restriction being that the unknown parameter lies in a finite parameter set. For a previously introduced scheme of identification and control based on "biased" maximum likelihood estimates, it is shown that 1) every Cesaro-limit point of the parameter estimates is "closed-loop equivalent" to the unknown parameter; 2) for both the discounted and long-term average cost criteria, the adaptive control law Cesaro-converges to the set of optimal control laws; and 3) in the case of the long-term average cost criterion, the actual cost incurred by the use of the adaptive controller is optimal and cannot be bettered even if one knew the value of the unknown parameter at the start.