Nash equilibria in partial-information games on Markov chains

We consider a two-player partial-information game on a Markov chain, where each player attempts to minimize its own cost over a finite time horizon. We show that this game has always a Nash equilibrium in stochastic behavioral policies. The technique used to prove this result is constructive but has severe limitations because it involves solving an extremely large bi-matrix game. To alleviate this problem, we derive a dynamic-programming-like condition that is necessary and sufficient for a pair of policies to be a Nash equilibrium. This condition automatically gives Nash equilibria when a pair of "cost-to-go" functions can be found that satisfy certain inequalities.