Sequential Necessary and Sufficient Conditions for Capacity Achieving Distributions of Channels With Memory and Feedback

We derive sequential necessary and sufficient conditions for any channel input conditional distribution P
_0,n
^=Δ
{P
_Xt|Xt-1,Yt-1
: t = 0, ..., n} to maximize the finite-time horizon directed information defined by C
_Xn→Yn
^{FB =Δ}
sup
_P0,n
I(X
ⁿ
→ Y
ⁿ
), where I(X
ⁿ
→ Y
ⁿ
) = Σ
_t=0
ⁿ
I(X
^t
; Y
_t
|Y
^t-1
), for channel distributions {P
_Yt
|Y
^t-1
,Xt: t = 0, ..., n} and {P
_Yt
|Y
_t-M
^t-1
,X
_t
: t = 0, ..., n}, where Y
^{t =Δ}
{Y
^-1
, Y
₀
, ..., Y
_t
} and X
^{t =Δ}
{X
₀
, ..., X
_t
} are the channel input and output random processes, and M is a finite non-negative integer. We apply the necessary and sufficient conditions to application examples of time-varying channels with memory to derive recursive closed form expressions of the optimal distributions, which maximize the finite-time horizon directed information. Furthermore, we derive the feedback capacity from the asymptotic properties of the optimal distributions by investigating the limit C
_X∞→Y∞
^{FB =Δ}
lim
_n→∞
(1/(n + 1))C
_Xn→Yn
^FB
without any á priori assumptions, such as stationarity, ergodicity, or irreducibility of the channel distribution. The framework based on sequential necessary and sufficient conditions can be easily applied to a variety of channels with memory, beyond the ones considered in this paper.