A Low Complexity Approach to Model-Free Stochastic Inverse Linear Quadratic Control

In this paper, we present a Model-Free Stochastic Inverse Optimal Control (IOC) algorithm for the discrete-time infinite-horizon stochastic linear quadratic regulator (LQR). Our proposed algorithm exploits the richness of the available system trajectories to recover the control gain
$K$
and cost function parameters
$(Q,R)$
in a low (space, sample, and computational) complexity manner. By leveraging insights on the stochastic LQR, we guarantee well-posedness of the Model-Free Stochastic IOC LQR via satisfaction of the
Certainty Equivalence
optimality conditions. The exact solution of the control gain
$K$
is recovered via a deterministic, low complexity Least Squares approach. Using
$K$
, we solve a completely model-free non-iterative SemiDefinite Programming (SDP) problem to obtain a unique (up to a scalar ambiguity)
$(Q,R)$
, in which optimality and feasibility are jointly ensured. Via derivation of the sample complexity bounds, we show that the non-asymptotic performance of the Model-Free Stochastic IOC LQR can be characterized by the signal-to-noise (SNR) ratio of the finite set of system state and input signals. We present a model-based version of the algorithm for the special case where
$(A,B)$
is available, and we, further, provide the extension to the Stochastic Model-Free IOC linear quadratic tracking (LQT) case.