Estimation entropy and its operational characteristics in information acquisition systems

We consider a pair of correlated processes in {Z
_n
}
_n=-infin

^infin
and {Z
_n
}
_n=-infin

^infin
, where the former is observable and the latter is hidden. The uncertainty in the estimation of S
_n
upon the finite past history of Z
₀

^n-infin1
is H(S
_n
|)Z
₀

^ninfin1
which is a sequence of n. The limit of Cesaro mean of this sequence is called the estimation entropy. We show that the estimation entropy is the long run average entropy of the belief state on the hidden process obtained from the observation process. Estimation entropy inversely measures the observability of the hidden process through the observed process, and its minimization is the goal for optimal observability problems such as sensor scheduling. In this paper we describe such an operational characterization of estimation entropy.