Distributionally Robust Optimal and Safe Control of Stochastic Systems via Kernel Conditional Mean Embedding

We present a distributionally robust framework for dynamic programming that uses kernel methods to design control policies satisfying both safety and optimality specifications. Specifically, we leverage kernel mean embedding to map the transition probabilities governing state evolution into an associated reproducing kernel Hilbert space. Our key idea lies in combining conditional mean embedding estimated from past data of system trajectories with the maximum mean discrepancy distance to construct an ambiguity set, and then design a robust control policy using techniques from distributionally robust optimization. The main theoretical contribution of this paper is to leverage functional analytical tools to prove that optimal policies for this infinite-dimensional min-max problem are Markovian. Additionally, we discuss approximation schemes based on discretization of inputs to make the approach computationally tractable. We validate the main theoretical findings of the paper in a benchmark control problem involving safe control of thermostatically controlled loads.